Fixes from the review of the desktop import.
- A message another program holds is no longer recorded as deleted. Only a
real absence moves the import past it; anything else is tried again and
then reported as a failure, and the next run reads it.
- A resume that can't read some folders names them in a toast with a choice
to stop trying, keeps the job open and retries only those at the next start.
Files chosen by hand in the same dialog are named, since they can't come
back by themselves.
- A wipe or sign-out releases the job in the shell before the record goes.
Leaving the dialog without starting releases the confirmed files. Job
records are per account, and a start is refused while another window holds
that account's job.
- Dedupe goes by the bytes without state headers: a reused Message-ID of the
same length dropped a real message. Older records still count, so nothing
goes in twice.
- A lost connection on a folder or Email/import pauses instead of counting
towards the failures that stop an import.
- The taskbar goes through `lib/desktop/taskbar-progress.ts`.
Test changes: the taskbar test now renders the host, because its helper in
`local-import-job.ts` is gone, and the dedupe tests use `check`, which replaced
`keyFor` and `has`.
The timers used to delete everything an account kept, unsent mail included.
They now follow the refused-login rule: sign out and hide, the next sign-in
brings it back, otherwise it goes after 30 days. A login without "Remember
me" that ends with the browser, an admin wipe and a wipe from settings don't
change.
- `wipeOnRefusedLogin` is now `wipeOnForcedSignOut`, since it also covers the
limits. The server's `SESSION_TOO_OLD` goes the same way, as `session`.
- A refused account beside one that stays signed in shows in the switcher as
needing sign-in, with the count and the date, and gets one toast.
Existing tests changed because the rule did: the session timer tests expected
a plain `logoutAll`, and the auth store tests expected no hook at the session
limit.
A refused login used to delete the account's data, and with it everyone's
S/MIME and OpenPGP keys, also with offline off. The usual cause is a password
changed on another device, which Stalwart 0.16 and 1.0 both answer with
`invalid_grant`, so that lost unsent mail for little protection.
- A refused login signs out at once and hides the data. A new password brings
it back, unsent mail included. Otherwise it goes after 30 days. The sign-in
page gives a count and a date. The 24-hour timer is gone.
- `organisation.devices.wipeOnRefusedLogin`, off by default, asks for deletion
at once, of that account's own data only.
- The session limit no longer counts as a refusal on any route. It hides the
data and says nothing.
- A wipe runs under a Web Lock, tells other windows when it starts and ends,
and queues its report before deleting anything. It reports only once the
deletion worked and no plugin database is left.
- More goes with a wipe: Pro tabs, search history, Files and WebDAV state,
templates, pop-out drafts, shown notifications, push state, and Web Push
with the last account.
- The unsent count is per account, and unknown rather than 0 when the store
can't say. A new row is checked against the account that asked, and a
sign-out drops its row.
Existing tests changed because the rule did: `auth-store-keep-local.test.ts`
expected a wipe on `invalid_grant` and at the session limit, and
`device-wipe.test.ts` expected shared keys to go on a refused login.
Until now a lost laptop kept its offline mail for as long as anyone could
open it, and an account whose login the server had ended kept its copy too.
- `lib/devices`, `app/api/devices`: browsers with offline on and desktop
apps register, in a durable collection with no expiry. A check-in goes
by a per-device secret, so a revoked device still hears its wipe.
- `app/api/admin/devices` and the org devices page: wipe one device or all
of a user's, with audit entries for each request and each confirmation.
- `lib/offline/device-wipe.ts`: a start finishes any wipe cut short, then
waits up to a second for the instance before showing the local copy. A
wipe takes the store, keys, plugin databases and handoffs, signs out,
and reports back, again at the next contact if the report failed.
- `lib/jmap/client.ts`, `stores/auth-store.ts`: only `invalid_grant`, the
session limit, or a 401 lasting past a re-auth count as refused. A 500,
a proxy page or an outage never do.
- Settings: "Remove everything from this device", with a confirmation.
`lib/shared-store/__tests__/import-data.test.ts` changes its assertion
because it lists every collection the import copies, and `devices` is new.
The lock is off by default, and an OS policy can require it and set the
timeout. It locks after a time without use counted across all windows and
tabs, on OS screen lock in the desktop app, and from Lock now.
Locking saves open composers, stops the outbox at its next message, closes
the store and stops its worker with every key, then reloads the page, so
nothing of the old page stays in reach. The new page holds the app back at a
Suspense boundary until someone unlocks. Pop-outs get no session while
locked, push notifications name nobody, and subjects leave plain storage.
One random lock key wraps each account's key as a layer. The PIN wraps the
lock key with PBKDF2 at 600,000 iterations, with a wait after wrong tries
that a reload doesn't reset. A passkey with PRF and the desktop's OS storage
wrap it too. Details and limits are in the decisions file.
A persisted account id with no subject yet read as decided: Everyone's
noOffline wiped mail a group scope allowed, and a wider scope went out at
every reload. A public, partial policy counted as the organisation's word,
and turned on the desktop default in a forbidding organisation.
- an organisation's cap on one file now binds a pin
- Clear and the off switch read what would be lost when asked, messages,
changes and drafts, and Clear waits for a connection
- turning one account off survives the last one going off, and off then on
no longer warns of an eviction
- a tab that takes over the store sends no scope until it has heard the policy
- a dropped account is judged by what applies to anyone
- a choice reaches the store once it settles; progress announces phases only
- restored Pro tabs ask for their titles
One attachment scope: the kept-attachments code, which takes chunks with a
file and spares pinned mail. A draft only on this device stays through a
tightened window. Mail's budget still gives way to what Files keeps.
The flush keeps one `outbox.dropOrphans` caller, the mail branch's daily
one with its 7 days, and drafts' own query says what's needed, so
`addHeldFileSource` and its test are gone. `unsentCount` counts local
drafts, and a draft a waiting send names counts once. The scope check in
`recordBlob` and the permanent-status rule moved to
`lib/store/mail/attachments.ts`, with tests.
Eviction now goes by the base's signal: `storeLost` and the `store-reset`
notice, with `bulwark-store-opened` as the marker and `prune-files` for files
with no key. The storage page and the lost count hang off that. What a user
picks is stored under `bulwark-offline-scope`, the key `offlineScopeFor`
reads, and `email-snapshot` follows the base's rule. `OfflineEnabled` is read
from the desktop bridge when it's there.
Review of the offline drafts found these.
- Every new message gets new part ids and old ones die with it, so an
open composer failed with `blobNotFound` after a flush. The client and
the flush re-read the draft and re-point parts once. The fake now
gives new part ids, which is why `drafts.test.ts` and
`drafts-client.test.ts` compare file contents instead of blob ids.
- A Message-ID alone could destroy another client's message. A copy now
has to be solely in Drafts, with `$draft`, close to the attempt, with
the same subject and size, and what the lookup saw is the destroy's
base. Ghosts ride on the replacing row and only queue once it's up.
`drafts.created` no longer queues a copy that arrived for a replaced
row, so its test now expects the next flush to do it.
- Turning offline off asks about device-only drafts too.
- A kept file can't read back empty: chunks of a kept file are left
alone, and `keptBlob` and `heldFile` refuse pieces that don't add up.
- A flush stopped by a passing failure tries again, the banner shows
waiting drafts, and under the flush lock a `sending` row counts as
left by a dead tab at once. A lookup refused for good no longer stops
the outbox.
- `FetchTransport` streams downloads and reports a broken transfer as
interrupted, so one bad file doesn't end the pass.
- Raw messages come after every file, only into free room, and a file
needing room pushes them out first.
- An old version in another login stays until the new one is up. An
online send of an unflushed stand-in still removes its submission.
The offline section of Settings > Backup & reset now shows, per account, the
space each part takes, the last sync, a first sync's progress with the time
left, the window and attachment choices, and clearing the account. All
conflicts, calendar and contacts too, are listed there and counted in the
banner.
- the organisation's caps beat the user's choice and the desktop default, and
every path that takes a scope goes through `effectiveScope`
- the desktop app keeps mail without asking, says so once, and stays off for
an account where the user turned it off
- `navigator.storage.persist()` is asked on turning it on, and a refusal
warns with a way to ask again; Safari gets the 7-day hint when not installed
- a store the browser deleted is noticed, says what was lost and syncs again
- a full disk says so, with what to do
- those who had offline mode keep it, with the 30 days they had
- Keep offline in the Files app's menu
A draft saved offline now stays on the device under a stand-in id and
goes to the server by itself when it's back. Until now every save was
refused and the composer showed "not saved".
- `lib/store/drafts`: one pending create per draft, however often it's
saved. A draft from the server edited offline is replaced by create,
then destroy, so a failure leaves a copy. Each create has a fixed
Message-ID, looked up in Drafts after a lost response. A refused
draft stays on the device with its error and can be opened. Changed
on another device: both are kept and the user is told.
- `JMAPClient`: stand-in ids in Email/get and Email/set resolve like
calendar's, also when answering from the device. A file attached
offline stays readable, and is uploaded before a save that names it.
- Held files and kept mail files are stored in 1 MB chunks. Orphans are
dropped at the end of a flush, counting what drafts need.
- `lib/store/mail/attachments.ts`: attachments, inline images and raw
messages per scope, newest first, after everything else in a pass,
with eviction that spares pinned mail and never sees held files.
- `mail.rawBlob` and `mail.rawStatus` for an offline export.
Changed test assertions: `outbox.heldFile` and `mail.getBlob` return
`blob` instead of `data`, so a file isn't held whole in memory. The
attachment tests ignore raw downloads, and mail-sync runs with
attachments off, because raw messages add a commit per message.
Any site could POST a form to the share target on load: with no clicks
the composer opened with its subject and file, and autosave stored a
real draft on the server 2 seconds later. The worker can't see where a
POST came from, and Chromium's link capturing accepts form submissions,
so an installed-app window proves nothing either.
- every share landing now asks first, in every window and browser; a
text landing address gets the same question. Nothing goes up before
the answer.
- handed-in text and files are the composer's starting state: no
autosave, page-close save or sign-out flush before the first real
edit.
- the worker ignores frames, keeps at most 5 entries and 100 MB for 15
minutes, sweeps on activate, on each share and from a page timer,
and seals what it keeps with a key passed in the landing fragment.
Offline wipes clear it too.
- the server fallback uses the fragment and caps by encoded length:
3000 Japanese characters made a 27 KB `Location` and a 431.
- a second compose handoff waits until the composer closes, also one
set aside by reading a message, where it used to replace the first.
- a launch leaving the mail page saves what was just typed first.
- an expired handoff or a missing stored file now says so; shares get
the control-character stripping and limits of `parseMailto`.
Existing tests changed because the landing moved from the query to the
fragment, stored entries gained a sealed part, and a share now asks
first.
The IndexedDB cache, its crypto, the responder and the page-side sync
go. lib/offline keeps its structure, policy, wipes, sign-out gates and
notices; answers, the outbox and sync now live in lib/store, in a worker
behind lib/store/client/api.ts.
What users get besides that:
- every folder is kept, not just the Inbox and opened ones
- delete for good and empty trash work offline
- a change the server refuses, or that meets a message changed on
another device, is shown instead of dropped
- a send whose response got lost is looked up by a Message-ID fixed when
it was queued, then finished, submitted or sent again
- with the store on, a start paints from the local copy, then connects
An account the app drops by itself no longer wipes its data: only a
sign-out or an explicit removal does. The outbox of the old database
moves over once, in lib/offline/migrate-legacy.ts, and the old database
goes only after the store committed. `isNetworkFailure` now counts
timeouts and proxy 5xx. Sends and outbox requests are never answered
from the device.
Tests moved with the code: cache.test.ts and responder.test.ts are
replaced by lib/store/__tests__/, and review-fixes.test.ts, outbox.test.ts
and client-fallback.test.ts now run the real engine on node:sqlite.
Changed assertions: an uncertain send is looked up rather than left alone
forever, destroy and empty trash are queued rather than refused, and #9
(opened folders) became "a folder the server deleted goes", since all
folders are kept.
A notification for one message gets buttons where the browser has room
for them, and a click on its body works as before. The service worker
has no login, so an open tab takes the action first, through the store
actions. With no tab open a new same-origin route, /api/push/action,
does Mark read and Archive with the context cookie. When the cookie is
gone, the token has expired or the login can't be told apart, the
message opens in the app instead. Reply opens the composer through a
`?reply=1` link, with any text typed on the notification.
- `public/sw.js`: buttons, asking open tabs, the route, `sync` handling
- `app/api/push/action`: same-origin, checks the login and the account,
honours the admin's Archive gate, files by the mirrored archive mode
- `lib/pwa-notify`: the page's side, the mirror for the worker, and
`requestOutboxSync`, which the offline queue calls
- Background Sync, thin version: a `sync` event tells open tabs to flush
Two push-preview-route assertions change: they match the whole answer
for a named message, which now also carries `login`, what the action
route checks so a stale notification can't act on whoever holds the
cookie slot now.
The installed app only handled mailto: and webcal: launches. The launch
consumer dropped files, and nothing could hand a File to the composer or
the import dialogs, because the pending mailto is JSON in sessionStorage.
- manifest: `shortcuts` named in the browser's language, `file_handlers`
for .eml, .ics and .vcf, and `share_target`, all with the base path and
per-domain branding. Lite's manifest is unchanged.
- `lib/launch-handoff.ts`: the one place files and links wait, in memory
and in the Cache API, so they survive a sign-in that leaves the page.
The desktop bridge can call the same functions, since launchQueue never
fires in Electron. Sign-out clears it.
- launch consumer: routes files by type, and takes a shortcut to an open
window, which focus-existing would otherwise only focus.
- .eml opens in a new host for the preview, also before sign-in, with the
strictest device policy deciding about saving. .ics and .vcf open their
import dialogs through a new `initialFile` prop. A .ics needs the server,
so offline the dialog says so and carries on once back online.
- composer: `initialFiles` goes through `addFiles`, so plugin vetoes and
size limits still apply, also from a Pro compose tab.
- `sw.js` takes the share POST before the offline check and keeps the files
for the page. Without a worker, `/protocol/share` passes text and links
on and the page says the files are lost. `protocol` joins the app
sections and the handler pages join the shell, so a mailto: launch
offline no longer gets a network error.
Browsers without launchQueue or a service worker behave as before.
The review of offline mode found ways to send a message twice, to lose
unsent mail without a word, and to send mail without its signing,
encryption or check plugin. This fixes them and the smaller findings:
- each outbox message is taken for sending in one database transaction
under a Web Lock per account; only a failure before the submission
request (SendNotStartedError from the client) puts it back, any other
one marks it "may have gone out"; read-backs after the submission no
longer fail a send; a send of a closed tab turns uncertain after a minute
- every sign-out the user didn't confirm names the deleted messages on
the sign-in page, or in a toast when another account stays signed in
- nothing is queued while an enabled plugin isn't running (after a start
without the network none can); the composer says why
- the outbox waits for a fresh policy and holds messages the rules have
since made stricter (a required label, a required send check)
- another tab switching offline mode off sticks: only switching it on
may create the database
- only attachments of kept messages are kept, filed with their message,
deleted with it and under the cap between syncs; file names and types
are sealed; folders that went away leave the scope, and a folder that
couldn't be asked keeps its mail without holding the others back
- a group scope that forbids offline mode waits until the account's
groups are known; one server down no longer holds another's mail
- an offline start only when the server is really unreachable, and the
real sign-in follows by itself once it answers
- the service worker files pages and chunks under the build that served
them, and the shell list of another build isn't kept under this one
- a raw send counts as the composer's only from the plugin running its
takeover, for that identity
- errors and refusals are in all languages; the banner tells "offline"
from "server can't be reached"; outbox buttons name their message;
changes and messages go in the order they were made
Linus Rath
Nick Sweeting