Commit Graph
66 Commits
Author SHA1 Message Date
Linus Rath 335e1edc10 Merge branch 'feat/desktop-offline' into feat/do-app-lock
# Conflicts:
#	components/settings/general-settings.tsx
#	hooks/use-calendar-alerts.ts
#	lib/desktop/app-lock-state.ts
#	lib/desktop/policy-schema.ts
#	lib/desktop/types.ts
2026-11-03 17:52:30 +01:00
Linus Rath dcfe25c754 Merge branch 'feat/do-store-export' into feat/desktop-offline 2026-11-03 17:39:54 +01:00
Linus Rath 530364b11c fix: stop a cancelled or wiped export from downloading, and say why an export stopped
Review of the local store export. A cancel, sign-out or wipe during the
final step still downloaded the file; now the sink takes the job's signal
and drops the Blob, and a job that outlasts the wait is forgotten when it
ends. Finished exports are in a list a later wipe or sign-out can empty,
also after the dialog closed.

- `localExportAllowed` uses `scopeDecided`, so a domain or group scope
  can't be missed before the subject arrives. It's checked before the
  save picker, in the same tick as the click.
- WebKit holds the final Blob in memory: exports stop at 1 GB, and a
  larger choice or one over `storage.estimate()` is refused up front.
- An empty folder that refuses during a full listing counts as empty,
  only when the store's own count agrees. Any other refusal stops the run.
- A picked file goes on an abort or an all-failed run; if it can't, the
  dialog names it. Failure texts come from our strings, not the browser's.
- A selection keeps to the chosen folders on a retry.

The test that the organisation shows as the reason now flips the policy
through the policy store, because the old one replaced the guard and so
never reached the watch that a real change goes through. The spec moves
to an account of its own, dso-export: other specs running at the same
time work in ent-dave, and its folder went missing there mid-run.
2026-11-03 17:29:42 +01:00
Linus Rath 3f264367af fix: harden the app lock: forget in an order that can't leave a sign-in without a PIN, hear wipes and clear subjects while locked, and refuse obvious PINs 2026-11-03 17:08:02 +01:00
Linus Rath 289cce9bd5 Merge branch 'feat/do-reminders' into feat/desktop-offline
# Conflicts:
#	lib/offline/offline-mode.ts
2026-11-03 16:58:11 +01:00
Linus Rath f21198b804 Merge branch 'feat/desktop-offline' into feat/do-app-lock 2026-11-03 16:24:34 +01:00
Linus Rath 71bd7af93d Merge branch 'feat/do-remote-wipe' into feat/desktop-offline 2026-11-03 16:20:09 +01:00
Linus Rath 3cda05a1fa feat: export mail kept on this device to mbox or EML zip, also offline 2026-11-03 15:40:57 +01:00
Linus Rath a52a39f9e6 fix: keep one bad event from stopping reminders, offer Join for web links only, and keep summaries until they are seen 2026-11-03 15:40:10 +01:00
Linus Rath 2836cb6466 Merge branch 'feat/desktop-offline' into feat/do-reminders
# Conflicts:
#	lib/desktop/types.ts
#	lib/offline/offline-mode.ts
#	lib/store/areas.ts
#	locales/ar/common.json
#	locales/ca/common.json
#	locales/cs/common.json
#	locales/da/common.json
#	locales/de/common.json
#	locales/en/common.json
#	locales/es/common.json
#	locales/et/common.json
#	locales/fa/common.json
#	locales/fr/common.json
#	locales/he/common.json
#	locales/hu/common.json
#	locales/it/common.json
#	locales/ja/common.json
#	locales/ko/common.json
#	locales/lv/common.json
#	locales/mn/common.json
#	locales/nb/common.json
#	locales/nl/common.json
#	locales/pl/common.json
#	locales/pt/common.json
#	locales/ro/common.json
#	locales/ru/common.json
#	locales/sk/common.json
#	locales/th/common.json
#	locales/tr/common.json
#	locales/uk/common.json
#	locales/vi/common.json
#	locales/zh-TW/common.json
#	locales/zh/common.json
2026-11-03 14:46:00 +01:00
Linus Rath 306b249c07 chore: merge the offline and desktop base into the app lock branch 2026-11-03 14:40:25 +01:00
Linus Rath 54852df738 test: stop a pinned download at the organisation limit when the size is missing 2026-11-03 14:12:19 +01:00
Linus Rath bd7d12f801 Merge branch 'feat/desktop-offline' into feat/do-remote-wipe
# Conflicts:
#	components/offline/offline-settings.tsx
#	locales/ar/common.json
#	locales/ca/common.json
#	locales/cs/common.json
#	locales/da/common.json
#	locales/de/common.json
#	locales/en/common.json
#	locales/es/common.json
#	locales/et/common.json
#	locales/fa/common.json
#	locales/fr/common.json
#	locales/he/common.json
#	locales/hu/common.json
#	locales/it/common.json
#	locales/ja/common.json
#	locales/ko/common.json
#	locales/lv/common.json
#	locales/mn/common.json
#	locales/nb/common.json
#	locales/nl/common.json
#	locales/pl/common.json
#	locales/pt/common.json
#	locales/ro/common.json
#	locales/ru/common.json
#	locales/sk/common.json
#	locales/th/common.json
#	locales/tr/common.json
#	locales/uk/common.json
#	locales/vi/common.json
#	locales/zh-TW/common.json
#	locales/zh/common.json
2026-11-03 13:58:22 +01:00
Linus Rath b9b31ebc5b Merge branch 'feat/desktop-offline' into feat/do-store-contacts-files
# Conflicts:
#	lib/store/area.ts
#	lib/store/engine/account-engine.ts
#	lib/store/engine/transport.ts
#	lib/store/files/index.ts
#	lib/store/files/keep.ts
2026-11-03 13:52:30 +01:00
Linus Rath 418bda4642 fix: stream kept files to the store, keep the old copy until the new one is in, and apply a tighter scope at once 2026-11-03 13:41:03 +01:00
Linus Rath 8523289a4b feat: let an admin wipe a lost device, and clear an account the server refuses for good
Until now a lost laptop kept its offline mail for as long as anyone could
open it, and an account whose login the server had ended kept its copy too.

- `lib/devices`, `app/api/devices`: browsers with offline on and desktop
  apps register, in a durable collection with no expiry. A check-in goes
  by a per-device secret, so a revoked device still hears its wipe.
- `app/api/admin/devices` and the org devices page: wipe one device or all
  of a user's, with audit entries for each request and each confirmation.
- `lib/offline/device-wipe.ts`: a start finishes any wipe cut short, then
  waits up to a second for the instance before showing the local copy. A
  wipe takes the store, keys, plugin databases and handoffs, signs out,
  and reports back, again at the next contact if the report failed.
- `lib/jmap/client.ts`, `stores/auth-store.ts`: only `invalid_grant`, the
  session limit, or a 401 lasting past a re-auth count as refused. A 500,
  a proxy page or an outage never do.
- Settings: "Remove everything from this device", with a confirmation.

`lib/shared-store/__tests__/import-data.test.ts` changes its assertion
because it lists every collection the import copies, and `devices` is new.
2026-11-03 13:19:44 +01:00
Linus Rath 099e3fa887 test: check calendar reminders in the browser on both servers: offline, one tab, system notification, missed summary 2026-11-03 13:09:37 +01:00
Linus Rath fb3c82eed5 feat: lock the app with a PIN, a passkey or Windows Hello, with mail out of memory and the local store closed
The lock is off by default, and an OS policy can require it and set the
timeout. It locks after a time without use counted across all windows and
tabs, on OS screen lock in the desktop app, and from Lock now.

Locking saves open composers, stops the outbox at its next message, closes
the store and stops its worker with every key, then reloads the page, so
nothing of the old page stays in reach. The new page holds the app back at a
Suspense boundary until someone unlocks. Pop-outs get no session while
locked, push notifications name nobody, and subjects leave plain storage.

One random lock key wraps each account's key as a layer. The PIN wraps the
lock key with PBKDF2 at 600,000 iterations, with a wait after wrong tries
that a reload doesn't reset. A passkey with PRF and the desktop's OS storage
wrap it too. Details and limits are in the decisions file.
2026-11-03 13:08:37 +01:00
Linus Rath 8d802e4601 fix: decide on offline policy only with the whole policy and the user's groups, and ask before losing edits
A persisted account id with no subject yet read as decided: Everyone's
noOffline wiped mail a group scope allowed, and a wider scope went out at
every reload. A public, partial policy counted as the organisation's word,
and turned on the desktop default in a forbidding organisation.

- an organisation's cap on one file now binds a pin
- Clear and the off switch read what would be lost when asked, messages,
  changes and drafts, and Clear waits for a connection
- turning one account off survives the last one going off, and off then on
  no longer warns of an eviction
- a tab that takes over the store sends no scope until it has heard the policy
- a dropped account is judged by what applies to anyone
- a choice reaches the store once it settles; progress announces phases only
- restored Pro tabs ask for their titles
2026-11-03 12:33:53 +01:00
Linus Rath 4fd5c8c1aa feat: keep a record of calendar reminders in the local store, so each fires once and missed ones can be summed up 2026-11-03 12:25:50 +01:00
Linus Rath b635986506 chore: merge drafts and kept attachments into the settings branch
One attachment scope: the kept-attachments code, which takes chunks with a
file and spares pinned mail. A draft only on this device stays through a
tightened window. Mail's budget still gives way to what Files keeps.
2026-11-03 12:04:47 +01:00
Linus Rath e215423b5b fix: notice a lost store also when the first try to open it had no session yet 2026-11-03 11:24:39 +01:00
Linus Rath e98741d604 fix: say a store was reset also when a new owner took over the open, and slow raw downloads to one a second 2026-11-03 11:17:53 +01:00
Linus Rath 60eef9c1ef chore: merge the reviewed mail branch, pop-outs and desktop core into the offline drafts branch
The flush keeps one `outbox.dropOrphans` caller, the mail branch's daily
one with its 7 days, and drafts' own query says what's needed, so
`addHeldFileSource` and its test are gone. `unsentCount` counts local
drafts, and a draft a waiting send names counts once. The scope check in
`recordBlob` and the permanent-status rule moved to
`lib/store/mail/attachments.ts`, with tests.
2026-11-03 10:40:52 +01:00
Linus Rath e0abc7fa18 chore: merge the reviewed store, pop-outs and desktop core into the settings branch
Eviction now goes by the base's signal: `storeLost` and the `store-reset`
notice, with `bulwark-store-opened` as the marker and `prune-files` for files
with no key. The storage page and the lost count hang off that. What a user
picks is stored under `bulwark-offline-scope`, the key `offlineScopeFor`
reads, and `email-snapshot` follows the base's rule. `OfflineEnabled` is read
from the desktop bridge when it's there.
2026-11-03 10:40:47 +01:00
Linus Rath 4fd82437ad fix: keep attachments alive through a flush and never destroy a copy that isn't ours
Review of the offline drafts found these.

- Every new message gets new part ids and old ones die with it, so an
  open composer failed with `blobNotFound` after a flush. The client and
  the flush re-read the draft and re-point parts once. The fake now
  gives new part ids, which is why `drafts.test.ts` and
  `drafts-client.test.ts` compare file contents instead of blob ids.
- A Message-ID alone could destroy another client's message. A copy now
  has to be solely in Drafts, with `$draft`, close to the attempt, with
  the same subject and size, and what the lookup saw is the destroy's
  base. Ghosts ride on the replacing row and only queue once it's up.
  `drafts.created` no longer queues a copy that arrived for a replaced
  row, so its test now expects the next flush to do it.
- Turning offline off asks about device-only drafts too.
- A kept file can't read back empty: chunks of a kept file are left
  alone, and `keptBlob` and `heldFile` refuse pieces that don't add up.
- A flush stopped by a passing failure tries again, the banner shows
  waiting drafts, and under the flush lock a `sending` row counts as
  left by a dead tab at once. A lookup refused for good no longer stops
  the outbox.
- `FetchTransport` streams downloads and reports a broken transfer as
  interrupted, so one bad file doesn't end the pass.
- Raw messages come after every file, only into free room, and a file
  needing room pushes them out first.
- An old version in another login stays until the new one is up. An
  online send of an unflushed stand-in still removes its submission.
2026-11-03 10:06:17 +01:00
Linus Rath e8696094e4 fix: open an account again in the owner tab after another tab cleared it 2026-11-03 10:05:43 +01:00
Linus Rath 7dc4c775cd fix: don't wipe a store just because this window can't open it, and don't send a stand-in when a file's upload is held again 2026-11-03 09:45:03 +01:00
Linus Rath fb40944c2d feat: hold kept files to the scope at once, and count them towards what each account takes 2026-11-03 09:28:31 +01:00
Linus Rath ebec09212e fix: say plainly when a private window, old stranded mail or a silent server is the problem, and clean the domain of a legacy Message-ID 2026-11-03 09:25:07 +01:00
Linus Rath 024078c597 fix: clear files held for nothing a week after a flush, leaving what other parts of the store still hold 2026-11-03 09:25:06 +01:00
Linus Rath 3dc2fc2c3c fix: give up on a kept attachment only when the server says it won't come 2026-11-03 09:25:05 +01:00
Linus Rath f2aedd2948 chore: merge contacts and files into the settings branch 2026-11-03 09:22:04 +01:00
Linus Rath 9b3bbc6070 fix: delete a store file whose key is gone, say that it happened, and stop waiting on an owner tab that doesn't answer 2026-11-03 09:18:41 +01:00
Linus Rath 5dc0f05a70 fix: count a draft with a waiting send once when signing out 2026-11-03 09:16:32 +01:00
Linus Rath 18efb9f4c3 fix: wait 30 seconds before looking up a send, bound the waits at a start, and keep opened files only as the chosen scope allows 2026-11-03 09:00:52 +01:00
Linus Rath 62a47434e3 fix: ask before a sign-out about everything unsent, including changes, and say when the local store won't open 2026-11-03 08:56:30 +01:00
Linus Rath a7c3c5b9a6 feat: prune kept mail at once when the scope tightens and fetch it back when it widens
The store took a scope only when it opened and acted on it only at a full
sync, so a lower cap left old mail on the device until something else
forced a resync. Now the engine applies a changed scope first, without the
network, and once a day as the window moves on.

Also tell the page whether an account's file was there when it opened, so
it can notice a store the browser deleted.

- `calendar-sync.test.ts` now syncs with `days: null`: its old events test
  checks the calendar's own two years, and the calendar follows the scope now
- `page-side.test.ts`: the fake worker answers an open with what it found,
  as the real one does
2026-11-03 08:56:24 +01:00
Linus Rath cdda75d13b fix: keep queued changes and mail through rate limits, logins that need renewing and proxy errors, and show the server's copy after a real refusal 2026-11-03 08:49:53 +01:00
Linus Rath 5c5d982f9d feat: save drafts and keep attachments while offline
A draft saved offline now stays on the device under a stand-in id and
goes to the server by itself when it's back. Until now every save was
refused and the composer showed "not saved".

- `lib/store/drafts`: one pending create per draft, however often it's
  saved. A draft from the server edited offline is replaced by create,
  then destroy, so a failure leaves a copy. Each create has a fixed
  Message-ID, looked up in Drafts after a lost response. A refused
  draft stays on the device with its error and can be opened. Changed
  on another device: both are kept and the user is told.
- `JMAPClient`: stand-in ids in Email/get and Email/set resolve like
  calendar's, also when answering from the device. A file attached
  offline stays readable, and is uploaded before a save that names it.
- Held files and kept mail files are stored in 1 MB chunks. Orphans are
  dropped at the end of a flush, counting what drafts need.
- `lib/store/mail/attachments.ts`: attachments, inline images and raw
  messages per scope, newest first, after everything else in a pass,
  with eviction that spares pinned mail and never sees held files.
- `mail.rawBlob` and `mail.rawStatus` for an offline export.

Changed test assertions: `outbox.heldFile` and `mail.getBlob` return
`blob` instead of `data`, so a file isn't held whole in memory. The
attachment tests ignore raw downloads, and mail-sync runs with
attachments off, because raw messages add a commit per message.
2026-11-03 08:49:19 +01:00
Linus Rath 55ca12adbd fix: say when a start stays on the local copy, queue its sends, paint it only with folders kept and fill the From list after the swap 2026-11-03 08:43:35 +01:00
Linus Rath 10a2c99b83 fix: keep syncing when a message has two keywords differing only in case, and compare keywords the way the server does 2026-11-03 08:22:15 +01:00
Linus Rath fcbc6edb89 fix: stop losing the first local store call of a tab that just became the owner, which left it syncing for good 2026-11-03 07:17:23 +01:00
Linus Rath 20b2e7054e chore: merge search, notification actions and calendar into the mail branch 2026-11-03 07:02:39 +01:00
Linus Rath 2eb835b0b6 feat: merge offline calendar
# Conflicts:
#	lib/jmap/offline-hooks.ts
2026-11-03 06:39:38 +01:00
Linus Rath a3819f6a1c fix: keep offline calendar writes in order, and stop them getting lost or doubled
Review of the calendar store found writes that went missing, went out
wrong, or appeared twice.

- Writes with different scheduling flags stay separate and go out in
  order. Merging them collapsed the two steps of an occurrence change, so
  attendees who had answered got no mail, and "don't notify" went out to
  everyone with a later RSVP.
- An edit made after a create's lost response is sent as an update. An
  import of a uid the server has updates what differs.
- `createCalendarEvent`, `batchCreateCalendarEvents` and
  `createCalendarTask` choose a uid before the first send, so a lost
  answer online doesn't duplicate.
- The client resolves stand-in ids before the network, which covers an
  open dialog once the server is back.
- A stale whole `recurrenceOverrides` map no longer erases overrides.
- A 4xx refuses that write and the queue goes on. `serverFail`,
  `rateLimit` or an answer with no verdict retry instead of deleting.
- "This and all following" is refused offline with a clear message.
- A full list only drops what the server says it doesn't have. Sync adopts
  a stand-in by uid. Reminder keys survive a stand-in becoming real.
  Accounts the session no longer lists are forgotten.

Changed tests: the one asserting a sticky scheduling flag is gone, it
locked in the bug. Reminder keys now use the account and uid. Two flush
assertions drop `sent: 1`, because sync reads a lost write back first. The
two-account id test uses a listed account, as unlisted ones are removed.
2026-11-03 06:32:53 +01:00
Linus Rath 3060c60778 fix: stop calling every browser unsupported for the local store 2026-11-03 06:24:31 +01:00
Linus Rath 4c242e6a0c fix: keep an account's contacts when only another account could answer 2026-11-03 06:20:40 +01:00
Linus Rath 9fbe33004b fix: send contact edits that remove entries and stop reading own edits as conflicts
An edit that removed an entry was dropped at flush, because a server
value with more keys counted as already applied. Other fixes in the same
path:

- the local copy after a flush is the server's read-back, never what was
  sent, since Stalwart 1.0 stores a data: URI as a blob
- a delete or a newer edit after a lost response no longer reads as a
  change on the server, and an edit made during a refused send stays
- a create only becomes a card with its uid if that card holds what was
  sent, otherwise it is a refused conflict
- a refused or failed flush no longer stops contacts syncing
- request limits come from the session
- answers wait for the first full pass
- paging goes by anchor, so a deleted card doesn't make the next one
  get skipped and swept

contacts.test.ts: "continues a stopped full sync" now checks the anchor
instead of position 2000, because the cursor no longer pages by position.
2026-11-03 06:20:39 +01:00
Linus Rath 2481f060b7 feat: merge offline search 2026-11-03 06:17:13 +01:00