The app now has two product flavors. `github` is the sideloaded APK as
before, with the same package name and its in-app updater. `play` is a
bundle for Google Play as `org.bulwarkmail.mobile`, without the updater
or the installer permission, since Play forbids apps updating themselves.
- `release-android.yml` builds the Play bundle in a second job, checks
its signer, package name and permissions, and keeps it as an artifact.
With `PLAY_SERVICE_ACCOUNT_JSON` set it uploads it to internal testing.
- The Play flavor uses the new Firebase project; the GitHub flavor keeps
the old one until the app re-registers its token when it changes.
- Widget clicks and the device-sync account type follow the package name.
- Photos come from the system picker without asking for media access,
and the unused microphone permission from expo-camera is removed.
- A manual workflow run without a tag no longer replaces the APK of the
latest release.
The app was locked to portrait in app.config.js and the hand-kept
AndroidManifest.xml. Android 16 ignores that lock on screens 600 dp and
wider, so tablets and foldables rotated into layouts that were never
tested. This removes the lock in both places, keeps React state across
rotation and split-screen resizes (`smallestScreenSize` joins
`configChanges`), and fixes what broke in landscape:
- the thread pager sizes its pages from its own width and snaps back to
the open message after a rotation instead of showing half a neighbour
- compose and the thread view share one keyboard tracker; a rotation
with the keyboard open closes it, since Android sends no new keyboard
frame and the old portrait one blanked the whole landscape composer
- in a short window with the keyboard up the composer hides the format
bar and tightens its header so a few lines of text stay visible
- inputs with suggestions or context around them opt out of Android's
full-screen landscape keyboard
- month views fit whole rows to the real height, keep the month across a
remount, and the month dropdown re-measures and caps its height
- the week grid's header follows the remounted list after a rotation
- the event sheet's actions stay on screen in landscape
- every RN Modal allows all orientations, or iOS forces portrait
With no Trash mailbox, or when Mailbox/get came back as an error, the
Delete action fell through to destroying the email. A shared account
without its own Trash lost mail permanently on one tap.
- only move to Trash; do nothing when there is no Trash or the lookup
failed
- check that Email/set actually updated the email
- show "(No Subject)" in the collapsed text instead of a blank line
- require unlock for Delete (Android 12+)
Widgets had 22dp corners, rounder than the launcher's own widgets and
the app's cards. They now use 16dp, the Pixel launcher's widget radius,
on the surface, its frame and the picker previews.
The picker previews still showed the layouts from before the fixes in
this series (Month's arrows, the Vacation switch, the old task rows and
triage header). Redrawn from the sample data at each widget's default
size, light theme, with the rounded corners and frame the launcher
draws.
The previous fix started the refresh as the app went to the background,
but it still stopped after its first request: fetch hands every response
over through a JS timer, and React Native pauses timers while the app's
activity is in the background, so the answer was only read once the app
came back. The widgets kept the previous account after a switch and
Home, and changes made in the app reached them late.
The refresh now runs as a headless task of its own,
BulwarkWidgetRefresh, started through BulwarkWidgetRefreshService. A
running headless task keeps the timers going, and the service keeps the
process from being frozen before the widgets are redrawn. Where Android
refuses to start the service the app refreshes in place as before. A
refresh started for a change also no longer joins one that began before
it.
Brings feat/device-sync into main: on Android, each account's address
books and calendars can sync both ways with the phone's Contacts and
Calendar apps (Settings -> Contacts / Calendar -> Sync to this
device), through a "Bulwark" Android account with a contacts and a
calendar sync adapter.
- native side: the account, both sync adapters, provider access scoped
to the account, push routing to the syncs, sync problem notices
- the sync engine in `src/device-sync`, run as a headless task
- the settings sections, sign-out and removal handling, translations
- design, limitations and verification in `docs/device-sync.md`
A sync that had to start React Native left a listener on the React
host until a context came up. When the start failed (a debug build
without Metro) or never finished, every retried sync added another
one, each holding its run, account and extras, for the life of the
process, and they all fired at once when a context finally came up.
The wait now ends once, and removes its listener, for whatever
reason: a ready context, a start task that faulted, or 10 minutes
(no sync or push job waits longer; JobScheduler stops them). A
context that comes up later runs nothing of it. The adapter's own
check of its run in the callback still covers a run it gave up
sooner.
The native push router and the push task decided separately whether a
push may carry mail, and disagreed. For an account entry that is not
an object, or a map that names no type, the router started the mail
task as possible mail, but the task's guard read the same payload as
"no mail" and returned, so new mail it announced was never looked for.
A push without a kind was judged by its map in the router but taken
for mail by the task.
Both now apply the same rules: anything but a StateChange, a map that
is missing, unreadable or names no type, an unreadable account entry,
or a mail type means the mail task runs. Only a readable StateChange
naming contact, calendar or other non-mail types skips it.
Checked with one table of payloads run against the router's decision
and the task's guard alike.
On main every push ran the mail task, whose legacy path notifies the
newest unread inbox mail. Device sync narrowed that to pushes naming
EmailDelivery, so contact and calendar changes never notify. But a
subscription made by an older build listens to Email and Mailbox until
the app is opened again, and a server without EmailDelivery reports
new mail only as those changes. Their new mail no longer showed a
notification while the app was closed.
Email and Mailbox now count as possible mail again, in the native push
router and in the push task alike. A push that names only contact or
calendar types still never takes the mail path.
Checked with the push router and push task tests: an Email or Mailbox
change starts the task and the legacy path notifies the message,
next to the syncs a mixed map asks for.
"Android account settings" opened Android's page for the Bulwark
account through `ACCOUNT_SYNC_SETTINGS` with the account as the
`account` extra. The Settings app of Android 12L and later passes only
`:settings:show_fragment_args` on to that page, which then found no
account and closed at once ("Account provided does not exist: null").
Since starting it succeeded, the fallback to the list of synced
accounts never ran, and the button seemed to do nothing.
The account now also goes in the fragment arguments, where those
Settings apps read it; older ones still read the plain extra.
Checked on the API 32 and API 34 emulators: the button opens the
account's "Account sync" page with its Contacts and Calendar switches.
The native side ANDs the account's scope onto each selection the engine
sends, as `(where) AND (scope)`. A selection such as `1) OR (1` closed
the scope's parenthesis itself: on the calendar tables, where
CalendarProvider adds no account filter of its own, an update or delete
built that way would have reached the events, attendees and calendars
of other accounts, e.g. the user's Google calendar. The engine only
sends constant selections, so this guards against bugs rather than
anything an outside app could send, which is what the scope is for.
- Selections and sort orders must balance their parentheses outside
quoted literals, and may not end the statement, carry comments or
query other tables (`SELECT`, `UNION`, `ATTACH`, `PRAGMA`).
- An update of a data row by id that writes `data1` without restating
the mimetype has the row's mimetype looked up: on a membership row
`data1` names a group, which must be the account's, as on insert.
Such writes by selection are refused.
- `data14`, a photo row's file id, is never written: another account's
file id would have let `readPhoto` read that account's photo.
Checked: 58 JVM tests (`gradlew :app:testDebugUnitTest`, three new), the
debug build, and every selection the engine and planners send against
the new rules.
The sync adapter logged only the SyncResult counters of a run, so a
failed sync showed up in logcat as `ioErrors=1` and nothing else. The
reason was in the run's report, and JS warnings from a headless run
don't reach logcat; finding out that a run failed on "Invalid column
raw_contact_is_read_only" took reading the app's storage. The log line
now carries the report's outcome, its message (up to 300 characters)
and the number of failed items.
Checked: 55 JVM tests (`gradlew :app:testDebugUnitTest`, one new) and
the debug build.
With device sync on, Android should hear about a change as soon as the
app does, not only at the next periodic sync. This adds what tells it:
- The app's own contact and calendar edits, StateChanges on the live
stream for `AddressBook`/`ContactCard` and `Calendar`/`CalendarEvent`,
and the app coming back to the foreground request a sync of that
account and authority. Requests within 5 s fold into one, and a
StateChange the engine caused itself (its own upload echoing back) is
skipped.
- While the app is closed, the account's push subscription carries
those types too. The FCM and UnifiedPush services hand every push to
the native router from f743ab0, which requests the syncs, and start
the mail push task only when a push has `EmailDelivery`; the task also
returns early for such pushes, so a contact change can never show the
"newest unread mail" notification.
- At launch, on foreground and when Android's accounts change, the app
checks Android's accounts against its own: an account removed in
Android Settings shows sync off and is not recreated, and one whose app
account is gone stops syncing and posts "Sign in again", keeping its
data.
The functions that turn sync on and off (`src/device-sync/app/lifecycle.ts`)
come with this; the settings that call them land in a later commit, so
nothing here runs before an account has device sync on.
Checked: the new store, lifecycle, trigger and push tests, `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit, and the
debug build.
The webmail's QR encodes a `bulwarkmail://pair?...` link, but nothing
registered that scheme: scanning it with the system camera or Google
Lens did nothing, and without a camera there was no way in at all.
- `bulwarkmail` is registered in `app.config.js` (iOS) and the committed
AndroidManifest. Links are parked in a small store and picked up by
the sign-in screen, or by "Add account" when already signed in, once
per launch.
- "Paste sign-in link" on the sign-in screen (and next to the server
step's scan hint) takes a link copied from the webmail.
- While pairing, the waiting screen names the webmail being paired with.
The engine plans its provider reads and writes in JS, but only native
code can reach ContactsProvider and CalendarProvider. This adds that
bridge to the BulwarkDeviceSync module, with the rest of what the engine
and the app need from it:
- `query`, `applyBatch`, `readSyncState` and `readPhoto` take the
`ProviderQuery`/`ProviderOp[]` JSON of `src/device-sync/types.ts`,
check it (`sync/ProviderOps.kt`) and run it for one Android account of
our type (`sync/ProviderScope.kt`, `sync/ProviderIo.kt`). Every
selection gets the account ANDed in, every row an op names by id or
points to as a parent must be the account's, and every URI carries
`caller_is_syncadapter`, so nothing the sync writes marks rows dirty or
schedules an upload. The calls run one at a time on the module's own
thread through an unstable provider client, so a long batch never
stalls other module calls and a provider that dies fails the call
instead of the app.
- `showSyncProblem` and `clearSyncProblem` post and clear the one
notification per account that needs the user, e.g. "Sign in again".
- `setPushRoutes` stores which Android accounts each JMAP account feeds,
for the push router in `sync/PushRouting.kt`; the push services start
using it in a later commit.
- A run now ends exactly once and refuses a report that comes after
that. A sync requested while a run is open makes the run ask for
another sync when it ends, because SyncManager drops such requests as
duplicates. `onPerformSync` lets nothing escape any more: an exception
there killed the whole process, UI included.
- `ensureAccount` no longer hands an existing Android account to another
app account (`conflict`): its rows and SyncState belong to the first.
Payloads cross as JSON strings, which the bridge moves much faster than
nested maps. `src/device-sync/native.ts` is the typed JS side. The fake
provider follows the rules the emulator runs turned up: contacts
`data1`..`data14` read back as text, events and calendars the sync
adapter inserts have DIRTY NULL, attendee rows are addressed by id, an
op by id expects one row, and a vanished parent fails with `assert`
while another account's fails with `scope`.
Checked: 54 JVM tests (`gradlew :app:testDebugUnitTest`), debug and
release (R8) builds, `npm run typecheck` and `npm test` on this commit,
and on the API 32 emulator through the debug receiver: inserts with
back-references and group memberships, SyncState, photo writes and
scaled reads, asserts and expected counts, both scope refusals, the
499-op yield boundary, `tooLarge` for a 1.3 MB batch, and the
notification.
AOSP Contacts, and the apps built on its account-type parser, check the
EditSchema in `xml/contacts.xml` and drop the whole account type when it
breaks one of their rules: every data kind with a type column needs
`<Type>` children, the other kinds must have none, and name, photo,
nickname, organization, note and group membership need maxOccurs="1".
The schema from ed2525e listed phone, email, postal, event and
relationship without types, so the editors would not have offered the
Bulwark account at all.
The kinds now list the types device sync maps to JSContact, and the
rules are written down in the file so the next edit keeps them.
Checked on the API 32 emulator (AOSP Contacts): with a Bulwark account
present, "Create new contact" offers it and opens the editor saving to
it.
Android syncs the contacts and calendars of other services through
accounts and sync adapters. This adds the Android side of device sync:
a "Bulwark" account type (`${applicationId}.account`, set in
build.gradle), an authenticator that sends Settings → Add account to
the app instead of asking for credentials, and one sync adapter each
for the contacts and the calendar provider.
An adapter does no syncing itself. It starts the BulwarkDeviceSync
headless JS task from the main thread, through the helper the push job
uses, waits for the task's report and hands it to SyncManager as the
SyncResult. The engine that will write the report runs in JS, so the
existing client stays the only owner of token refresh, mTLS and
session discovery; for now the task only reports that device sync is
off. Nothing in the app creates an account yet, so the feature stays
unreachable until its settings land. Debug builds get an adb hook for
the device tests (DeviceSyncDebugReceiver, shell only).
Hard errors are reported without progress: SyncManager retries a
failed sync at once when it made progress, which would loop an auth
failure.
Checked on the API 34 and API 32 emulators with the debug build, from
logcat and `dumpsys content`:
- app in the foreground: report after 319-334 ms
- app in the background: 320-327 ms
- process killed: SyncManager starts it for the sync service, React
Native starts headless, and the report arrives after 16.2 s (API 34)
and 8.5 s (API 32), most of it the debug bundle download from Metro
- an io report is retried with back-off and an auth report is not
retried; too many deletions shows the system's "Attempted to delete
too many Contacts." notification
- with Metro unreachable the adapter gives up after 2 s instead of
waiting for its deadline
- `gradlew :app:testDebugUnitTest`: 10 JVM tests for the report mapping
The code that starts React Native for a headless JS task, when the
process was woken without any UI, lived inside BulwarkPushJobService.
The device sync adapters need the same start, so it moves to a shared
HeadlessJs object, with a small startTask helper next to it.
Push behaviour does not change:
- onStartJob still calls the start code on the main thread, and the
listener-then-ready-check order is the same code, moved verbatim
- the task still starts from `UiThreadUtil.runOnUiThread`, its id is
still recorded in that same UI-thread turn, and a refusal because the
app came to the foreground still logs and finishes the job
- the start code now returns React Native's start task, so a caller can
notice a start that fails (a debug build with no Metro); the push job
ignores it
The helper documents that it must run on the main thread: the ReactHost
is created on first use without a lock, so a background caller racing
MainActivity could create a second one.
Checked with `gradlew assembleDebug -PreactNativeArchitectures=x86_64`.
The splash background was #09090b in both modes, so in light mode a
cold start went from a near-black splash to the white first frame.
It is now #ffffff by day and #09090b at night (`values-night`), as
`splash.backgroundColor` and `splash.dark.backgroundColor` in
app.config.js already ask for, so a later `expo prebuild` writes the
same colours. Checked on the API 35 emulator: light mode stays white
from splash to login screen, dark mode is unchanged.
Every background push kept the app awake for the full 30 s timeout of
the BulwarkPushTask headless task, however quickly the task was done.
AppRegistry reports a finished task through the native
HeadlessJsTaskSupport module, and React Native 0.81 registers that
module only for the old bridge (CoreModulesPackage); the bridgeless
runtime's CoreReactPackage leaves it out. JS could never say it was
done, so the service, or the job, and its wake lock waited for the
timeout. BulwarkHeadlessJsTaskSupportModule provides the module under
the same name when the app runs bridgeless.
The push path also leaked a wake lock now and then. startHeadlessTask
called HeadlessJsTaskService.acquireWakeLockNow on the FCM worker
thread while the service took the same static lock on the main thread.
The method is not thread-safe, so both could create a lock, and the one
that was overwritten stayed held. The FCM thread now takes its own
wake lock, timed at 10 s, and the service drops it as soon as it holds
React Native's.
Measured on the API 35 emulator with the R8 release build:
- push handled by the job: 30,000 ms before, 75-155 ms now
- push handled by the service: the service ran 31 s before, it is
gone within a second now
- `com.facebook.react.i` wake lock still held 5 s after a push: 2 of
4 pushes before, 0 of 8 now
A push that arrived while the app was in the background started
BulwarkPushTaskService with `startService`. Android 8+ allows that only
while the app is temporarily allowlisted, which a high-priority FCM
message grants. The relay does send FCM data messages at HIGH priority,
but FCM delivers some at normal priority anyway (it downgrades apps
whose high-priority messages rarely end in a notification), and a
UnifiedPush distributor may not raise the app at all. The call then
threw `BackgroundServiceStartNotAllowedException` and the app crashed,
from the FCM thread or, for UnifiedPush, on the main thread.
The service start is kept as the first choice, so a high-priority push
works as before. When Android refuses it, the same headless task now
runs from a JobScheduler job (BulwarkPushJobService): expedited on
Android 12+, which runs at once while the app has expedited-job quota
and as a regular job after that, and a regular job on Android 8-11.
JobScheduler is a platform API, so nothing is added to the build;
WorkManager is not in the dependency tree. A generic "new mail"
notification built natively was the other option, but the relay's
payload cannot tell new mail from other changes and the native side
knows neither the notification settings nor what was already shown.
Checked on the API 35 emulator with the R8 release build, the process
killed and the push delivered past the allowlist:
- FCM data message: before, FATAL EXCEPTION on
Firebase-Messaging-Intent-Handle; now the job is scheduled 18 ms
after the refusal and runs the task
- UnifiedPush message (connector 3.3.5, sent the way a distributor
does): before, FATAL EXCEPTION on main; now the same job path
- allowlisted FCM message: still starts BulwarkPushTaskService
On a cold start the splash was followed by about a second of blank
white, then mail rows whose header sat under the status bar until it
popped in and pushed the list down.
- `AppTheme` had no `android:windowBackground`, so while JS loaded the
window showed the platform default (near-white, or grey at night).
It is now the app's own background, #ffffff by day and #09090b at
night (`values-night`), matching `src/theme/tokens.ts`.
- `NavigationContainer` had no theme, so React Navigation painted its
light default behind and between screens even in dark mode. It now
uses the palette's background.
- The mail list took its top inset from the native `SafeAreaView`,
which on Fabric only learns the insets a few frames after mounting.
It now pads with `useSafeAreaInsets()`, which the navigator's
safe-area provider fills from the initial window metrics on the first
render; the value is the same, it just arrives in time.
The notification only carried the local account it was posted for, so
tapping one for a message in a group or shared mailbox opened the
thread against the user's own JMAP account, where the message doesn't
exist.
The background task now passes the JMAP account the message was
fetched from as `jmapAccountId`. The native module stores it as an
intent extra, validated like the other ids, and hands it back with the
tap. `App.tsx` routes the open with it, the way the unified inbox does
for group messages. Notifications posted before this change have no
such extra and open against the account's own mail as before.
Most apps start an email with `ACTION_SENDTO` and a `mailto:` URI. The
manifest offered the app for it, but React Native's Linking only
forwards VIEW intents, so the app opened on the inbox instead of the
composer.
MainActivity now rewrites a `SENDTO mailto:` intent into the equivalent
`VIEW mailto:` link before React Native sees it, on a cold start and
when the running app receives it. The EXTRA_EMAIL, EXTRA_CC, EXTRA_BCC,
EXTRA_SUBJECT and EXTRA_TEXT extras are folded into the link's query.
Bcc recipients of a mailto link now reach the composer too; they were
dropped for VIEW links as well.
Sharing a photo or document from another app into Bulwark failed every
time: the upload task was handed the share's `content://` URI, but it
can only stream from a file path, so it failed with "Directory for
'/external/images/media/…' doesn't exist". The attachment was also named
after the URI's last segment (a MediaStore row id such as "1000000027"),
and SEND_MULTIPLE shares only carried a wildcard type like `image/*`.
- `uploadBlob` copies `content://` URIs into the cache, uploads the copy
and deletes it afterwards, also when the upload fails
- `ShareIntentStore` asks the providing app for the display name, size
and MIME type when the intent arrives, falling back to the URI segment
plus an extension for the MIME type
- the composer gets the real name, type and size, so the size limit
check now applies to shared files too
CI:
- .github/workflows/release-android.yml builds APK on release publish
and via workflow_dispatch, attaches it to the release.
- android/app/build.gradle: optional release signingConfig driven by
Gradle properties (ANDROID_KEYSTORE_BASE64 secret etc.); falls back
to the debug keystore when no secrets are set.
- .gitignore: untrack /android so the runner can build directly;
ignore build outputs (build/, .cxx/), Metro/Expo dev logs, scratch.
Track android/ sources including BulwarkFcmPackage.kt and
google-services.json (Firebase Android keys are public per Google's
docs and intended to ship in the APK).
JS sync: push notifications + background task, vacation responder
(api + store), contact activity component, search utilities, settings
polish, calendar/contacts/email store and screen tweaks.
Tooling: scripts/poll-reload.mjs — Windows polling-watcher that hits
Metro /reload on file change (workaround for unreliable fs.watch
without Watchman).
Linus Rath
arnauda-gh