Review of the calendar store found writes that went missing, went out
wrong, or appeared twice.
- Writes with different scheduling flags stay separate and go out in
order. Merging them collapsed the two steps of an occurrence change, so
attendees who had answered got no mail, and "don't notify" went out to
everyone with a later RSVP.
- An edit made after a create's lost response is sent as an update. An
import of a uid the server has updates what differs.
- `createCalendarEvent`, `batchCreateCalendarEvents` and
`createCalendarTask` choose a uid before the first send, so a lost
answer online doesn't duplicate.
- The client resolves stand-in ids before the network, which covers an
open dialog once the server is back.
- A stale whole `recurrenceOverrides` map no longer erases overrides.
- A 4xx refuses that write and the queue goes on. `serverFail`,
`rateLimit` or an answer with no verdict retry instead of deleting.
- "This and all following" is refused offline with a clear message.
- A full list only drops what the server says it doesn't have. Sync adopts
a stand-in by uid. Reminder keys survive a stand-in becoming real.
Accounts the session no longer lists are forgotten.
Changed tests: the one asserting a sticky scheduling flag is gone, it
locked in the bug. Reminder keys now use the account and uid. Two flush
assertions drop `sent: 1`, because sync reads a lost write back first. The
two-account id test uses a listed account, as unlisted ones are removed.
The providers test now expects the palette option, because the palette keeps its paging apart from the mail list's. The search-merge test for a position with no cursor changed too: it used to expect that position sent to the server, which skips rows after a merge.
The IndexedDB cache, its crypto, the responder and the page-side sync
go. lib/offline keeps its structure, policy, wipes, sign-out gates and
notices; answers, the outbox and sync now live in lib/store, in a worker
behind lib/store/client/api.ts.
What users get besides that:
- every folder is kept, not just the Inbox and opened ones
- delete for good and empty trash work offline
- a change the server refuses, or that meets a message changed on
another device, is shown instead of dropped
- a send whose response got lost is looked up by a Message-ID fixed when
it was queued, then finished, submitted or sent again
- with the store on, a start paints from the local copy, then connects
An account the app drops by itself no longer wipes its data: only a
sign-out or an explicit removal does. The outbox of the old database
moves over once, in lib/offline/migrate-legacy.ts, and the old database
goes only after the store committed. `isNetworkFailure` now counts
timeouts and proxy 5xx. Sends and outbox requests are never answered
from the device.
Tests moved with the code: cache.test.ts and responder.test.ts are
replaced by lib/store/__tests__/, and review-fixes.test.ts, outbox.test.ts
and client-fallback.test.ts now run the real engine on node:sqlite.
Changed assertions: an uncertain send is looked up rather than left alone
forever, destroy and empty trash are queued rather than refused, and #9
(opened folders) became "a folder the server deleted goes", since all
folders are kept.
api.email.getThread fetches only the newest 50 messages of a thread
(Thread/get, then Email/get for those ids) instead of all of them with
their bodies. api.email.getThreadCount answers the number of messages
from Thread/get, with its own rate limit.
plugin-host-thread.test.ts now expects getThreadEmails to be called with
{ newest: 50 }, the new option that limits what is fetched.
The review of offline mode found ways to send a message twice, to lose
unsent mail without a word, and to send mail without its signing,
encryption or check plugin. This fixes them and the smaller findings:
- each outbox message is taken for sending in one database transaction
under a Web Lock per account; only a failure before the submission
request (SendNotStartedError from the client) puts it back, any other
one marks it "may have gone out"; read-backs after the submission no
longer fail a send; a send of a closed tab turns uncertain after a minute
- every sign-out the user didn't confirm names the deleted messages on
the sign-in page, or in a toast when another account stays signed in
- nothing is queued while an enabled plugin isn't running (after a start
without the network none can); the composer says why
- the outbox waits for a fresh policy and holds messages the rules have
since made stricter (a required label, a required send check)
- another tab switching offline mode off sticks: only switching it on
may create the database
- only attachments of kept messages are kept, filed with their message,
deleted with it and under the cap between syncs; file names and types
are sealed; folders that went away leave the scope, and a folder that
couldn't be asked keeps its mail without holding the others back
- a group scope that forbids offline mode waits until the account's
groups are known; one server down no longer holds another's mail
- an offline start only when the server is really unreachable, and the
real sign-in follows by itself once it answers
- the service worker files pages and chunks under the build that served
them, and the shell list of another build isn't kept under this one
- a raw send counts as the composer's only from the plugin running its
takeover, for that identity
- errors and refusals are in all languages; the banner tells "offline"
from "server can't be reached"; outbox buttons name their message;
changes and messages go in the order they were made
Settings > Backup & reset gets "Keep mail on this device", off until the
user turns it on in this browser. While it is on:
- the JMAP client hands the server's answers to an encrypted cache in
IndexedDB (AES-GCM, non-extractable key) and answers from it when the
network is gone: the folder list, the last 30 days of the Inbox and of
the folders the user opened, bodies and attachments, at most 250 MB,
oldest out first; a background sync keeps it current
- read, flag, tags and moves made offline are queued and sent in order
when the server answers again; deleting for good and emptying a folder
are refused offline
- the composer, quick reply and the S/MIME and PGP plugins' sends that
complete the composer's go to an outbox once every step up to the
transport ran (checks, label, signature, plugin send checks); only the
finished send waits, with files attached offline
- the service worker keeps this build's app shell, so the app also
starts offline from the kept session and the last policy
- a full sign-out, the idle sign-out and signing one account out delete
it, a sign-out the user starts asks first when mail is unsent, and the
login page names what an automatic sign-out deleted; a policy fetch
that forbids it (devices.noOffline) wipes that account
Not in Lite.
Organisation > Devices now sets policy.json `organisation.devices`, each
switch off by default and scoped per domain or group:
- No downloads: attachments and files only preview (reader, conversation
view, Pro tab, list chips, .eml previews, the Files app). Download,
Download all, Open in new tab, Save to Files, drag-out, the browser's
PDF viewer (pdf.js instead) and media download controls go away.
- No export: no .eml export, View source, message drag-out, or folder and
account export; a running export stops and its file is deleted when
the switch turns on. Import stays.
- No printing: no Print buttons in mail, calendar and contacts; Ctrl+P
prints a note; the reader frame prints blank.
- No install prompt: no card, and the browser's own banner is held back.
- Watermark: the user's address and the date over the reader,
previews and every printout (mail, calendar, contacts, plugins).
A message follows its own account's policy; what isn't about one message
follows the strictest signed-in account. Every user path hides its
button, and the anchor and new-tab helpers refuse as a backstop
(lib/download-guard.ts), never the blob readers. The plugin APIs
`ui.downloadFile` and `ui.print` refuse too, and the office editor frame
loses `allow-downloads`.
Two assertions in components/mail/__tests__/blob-routing.test.tsx now
expect `{ kind: 'export' }` as the fifth argument of downloadBlob for an
.eml export: the export says what it is, so the backstop checks "No
export" rather than "No downloads" for it.
A message that failed for a passing reason (a dropped connection, a
server error, an expired token) used to count as done, so continuing
the import skipped it for good, and a slow upload followed by a cancel
imported the messages after it twice.
- The checkpoint keeps every message that is done, also past a gap,
and the next run of the same import tries only the rest again.
- Ten failures in a row, a full mailbox (overQuota) or a refusal
(forbidden) stop the import with a message instead of going through
the whole file.
- An upload without an answer pauses the import instead of failing the
message: 30 seconds, then until the upload window ends.
- Email/import, the export's Blob/get and folder listing pass the
client's own 429 pause by, and a 429 on them doesn't pause the rest
of the app.
- Signing out stops the job of that account only, writes no checkpoint
and imports nothing more afterwards; another account's sign-out
leaves the job alone.
- One tab imports a file at a time (Web Locks), and Import waits until
the dialog knows whether an earlier run can be continued.
The import dialog tests now wait for the Import button to be enabled
before clicking it, since it stays off during that lookup.
The participant availability test that expected "Free/busy isn't
available on this server" when Stalwart refuses the directory listing
now expects the attendee list: Stalwart 1.0.1 still answers
getAvailability for the principals the user knows with directory
queries off, so the list stays and names who couldn't be looked up.
The 1.0 build that refuses the lookups too keeps its own case.
Admins had logs and nothing else: client errors only reached the
browser console. An admin can now set a Sentry-protocol address (DSN)
for Bugsink, GlitchTip or Sentry under System > Monitoring, or with
`ERROR_REPORTING_DSN`, `ERROR_REPORTING_DSN_FILE`,
`ERROR_REPORTING_SAMPLE_RATE` and `ERROR_REPORTING_ENVIRONMENT`.
Bulwark Lite reads the same from `errorReporting` in policy.json.
Everything stays off while no address is set.
- browsers report errors from the error boundaries, window.onerror,
unhandled rejections and failed JMAP requests (method and error type
only), with route and JMAP method breadcrumbs and the chunks' debug IDs
- the server reports what Next hands to onRequestError and what routes
log with logger.error
- every report is scrubbed first: quoted text, addresses, URLs, file
names, tokens and the rest after the first line are removed, and the
server scrubs what browsers post to /api/errors again
- reports are deduplicated and rate-limited per session and process
- errors are counted per day for "errors today" on the admin page
- startup messages and audit entries go through the logger, so
`LOG_FORMAT=json` covers them
api.tasks.create and update take links by id (an http(s) URL, a mailto:
address or a path of this Bulwark, stored as its full URL; at most ten)
and relatedTo (by a mail's Message-ID or another object's UID, with its
relation types; at most ten). They merge on update as alerts do: an
entry given is set, one given null is removed, and entries the plugin did
not name, such as an attachment's link another app wrote, stay as the
server holds them.
TaskView returns both as any app wrote them. Lengths, ids and URL
schemes are checked as the rest of the calendar API checks them. Task
reads now ask the server for links too.
api.tasks.create and update take alerts by id: a trigger from the task's
due (or start) as an ISO 8601 duration, or at an instant in UTC; display
or email; at most five. An update merges by id, each alert written by its
JMAP patch path: an alert given is set, one given null is removed, and
alerts the plugin did not name, such as one another app wrote, stay as
the server holds them; null for the whole map removes every alert.
Whether the map is patched or written whole is decided on the task as the
server holds it, not an older copy. An alert kept with the same trigger
and action stays acknowledged, so a snooze is an update of its trigger,
which then fires again. While a task uses its calendar's default alerts,
a change makes them the task's own under new ids, as the task form does,
an alert under a default's id is refused rather than dropped, and
removing every alert turns the defaults off too.
TaskView returns the alerts, each with when it was acknowledged, and
useDefaultAlerts. api.tasks.acknowledgeAlert dismisses one alert (or
undoes that with null) by writing only its acknowledged. A plugin's
writes to one task go one after another, since Stalwart refuses a set
while another is under way on the same object. The task store applies
patch paths to the task the caller read, so a task the list does not
hold is shown and reported whole.
Message ids repeat across accounts, so a hook payload naming only an
emailId could not tell a plugin which mailbox the message lies in when
it is a shared or group account's.
onBeforeEmailStarToggle and onEmailStarToggle now carry accountId when
the starred message is not in the user's own account, as the
message-row-actions click context does. Every star (the row's button,
the reader's star, the key, the menu) goes through toggleStar, one
message at a time, so one accountId names it; the account is resolved
before the plugins are asked. An own message's payload is unchanged.
onAfterEmailSend now carries accountId, the JMAP account the sent
message lies in (a mail sent from a shared or group identity lies in
that account), so api.email.get finds it. sendEmail returns that account
as emailAccountId, the S/MIME send too. The hook also carries
composerId, the id the sending composer's slots and actions gave
plugins, so a plugin can act on what the user chose in that composer.
Linus Rath
Nick Sweeting
Jacob Zeitz
VasilisNtovolos