Commit Graph
2162 Commits
Author SHA1 Message Date
Linus Rath 20b2e7054e chore: merge search, notification actions and calendar into the mail branch 2026-11-03 07:02:39 +01:00
Linus Rath f2356cdfc7 fix: look a send up soon after its response got lost, also with the server still there 2026-11-03 06:40:45 +01:00
Linus Rath 2eb835b0b6 feat: merge offline calendar
# Conflicts:
#	lib/jmap/offline-hooks.ts
2026-11-03 06:39:38 +01:00
Linus Rath a3819f6a1c fix: keep offline calendar writes in order, and stop them getting lost or doubled
Review of the calendar store found writes that went missing, went out
wrong, or appeared twice.

- Writes with different scheduling flags stay separate and go out in
  order. Merging them collapsed the two steps of an occurrence change, so
  attendees who had answered got no mail, and "don't notify" went out to
  everyone with a later RSVP.
- An edit made after a create's lost response is sent as an update. An
  import of a uid the server has updates what differs.
- `createCalendarEvent`, `batchCreateCalendarEvents` and
  `createCalendarTask` choose a uid before the first send, so a lost
  answer online doesn't duplicate.
- The client resolves stand-in ids before the network, which covers an
  open dialog once the server is back.
- A stale whole `recurrenceOverrides` map no longer erases overrides.
- A 4xx refuses that write and the queue goes on. `serverFail`,
  `rateLimit` or an answer with no verdict retry instead of deleting.
- "This and all following" is refused offline with a clear message.
- A full list only drops what the server says it doesn't have. Sync adopts
  a stand-in by uid. Reminder keys survive a stand-in becoming real.
  Accounts the session no longer lists are forgotten.

Changed tests: the one asserting a sticky scheduling flag is gone, it
locked in the bug. Reminder keys now use the account and uid. Two flush
assertions drop `sent: 1`, because sync reads a lost write back first. The
two-account id test uses a listed account, as unlisted ones are removed.
2026-11-03 06:32:53 +01:00
Linus Rath 9e89fb9038 feat: merge notification actions and background sync 2026-11-03 06:28:25 +01:00
Linus Rath 51c09a1a69 fix: never let a notification's buttons go round plugins, session limits or Trash
Review of the notification actions. With a plugin that may stop a move
or read change, or a silent tab holding the login, the message now
opens instead of the route acting. The route answers 401 under a
session limit, and neither path touches a message since moved to Trash
or Junk. A tab that thawed declines an old click, Archive follows the
gate of the notification's login and files into a shared-out Archive
as the app's button does, Reply opens a composer in the Pro shell, the
route retries once when another client made the folder first and
leaves no empty folder behind, and logins match however the server
address is written.

Two assertions of this feature's own tests change: the route answers
401, not 403, when a login's scopes can't be worked out, since that now
also covers session limits; a tab's message to the page now carries
`at`.

The Pro shell's mail surface now reads a parked link's query from the
handoff: the address bar may have none by its first render, which lost
`?slot=` and `?owner=` there too.
2026-11-03 06:28:04 +01:00
Linus Rath 3060c60778 fix: stop calling every browser unsupported for the local store 2026-11-03 06:24:31 +01:00
Linus Rath 2481f060b7 feat: merge offline search 2026-11-03 06:17:13 +01:00
Linus Rath d44738af87 fix: keep deleted mail out of merged search results and keep paging right after a refresh
The providers test now expects the palette option, because the palette keeps its paging apart from the mail list's. The search-merge test for a position with no cursor changed too: it used to expect that position sent to the server, which skips rows after a merge.
2026-11-03 06:16:45 +01:00
Linus Rath 41987cb5cf fix: stop hostile HTML stalling mail sync and index each message only once 2026-11-03 06:16:44 +01:00
Linus Rath a025b78ed4 test: pin the fixed Message-ID, the found copy and the conflict list 2026-11-03 06:16:09 +01:00
Linus Rath 9271f0a6ef fix: never lose a message when a full sync runs across deletions, by paging on an anchor and asking the server before dropping anything
An existing assertion changes: resuming a full pass now starts from the last id seen rather than a position, since positions shift with every deletion.
2026-11-03 06:16:08 +01:00
Linus Rath f6db7ea53a fix: keep pinned and unread-first lists off a full sort, and stop syncing when the disk is full 2026-11-03 06:02:10 +01:00
Linus Rath e38b023ee3 chore: merge the store's speed work into the mail branch 2026-11-03 05:59:37 +01:00
Linus Rath 687e7cc012 feat: move offline mode onto the encrypted local store
The IndexedDB cache, its crypto, the responder and the page-side sync
go. lib/offline keeps its structure, policy, wipes, sign-out gates and
notices; answers, the outbox and sync now live in lib/store, in a worker
behind lib/store/client/api.ts.

What users get besides that:
- every folder is kept, not just the Inbox and opened ones
- delete for good and empty trash work offline
- a change the server refuses, or that meets a message changed on
  another device, is shown instead of dropped
- a send whose response got lost is looked up by a Message-ID fixed when
  it was queued, then finished, submitted or sent again
- with the store on, a start paints from the local copy, then connects

An account the app drops by itself no longer wipes its data: only a
sign-out or an explicit removal does. The outbox of the old database
moves over once, in lib/offline/migrate-legacy.ts, and the old database
goes only after the store committed. `isNetworkFailure` now counts
timeouts and proxy 5xx. Sends and outbox requests are never answered
from the device.

Tests moved with the code: cache.test.ts and responder.test.ts are
replaced by lib/store/__tests__/, and review-fixes.test.ts, outbox.test.ts
and client-fallback.test.ts now run the real engine on node:sqlite.
Changed assertions: an uncertain send is looked up rather than left alone
forever, destroy and empty trash are queued rather than refused, and #9
(opened folders) became "a folder the server deleted goes", since all
folders are kept.
2026-11-03 05:54:39 +01:00
Linus Rath 7aa12d19b5 chore: fail a Lite build that ships the local store's engine 2026-11-03 05:54:37 +01:00
Linus Rath 062a078e1e feat: keep the local store's engine for a start without a network 2026-11-03 05:54:36 +01:00
Linus Rath 29778da2d4 fix: wait out an unreachable token endpoint instead of counting it toward a sign-out
An existing assertion changes: a token endpoint that can't be reached now answers 503, not 500, because the client counts repeated 500s as permanent and signs the user out.
2026-11-03 05:54:34 +01:00
Linus Rath 4ee918a98f fix: list a large folder off its index and sync message bodies in bigger batches 2026-11-03 05:46:55 +01:00
Linus Rath 877a7a93b2 feat: keep calendars on the device and queue calendar changes while offline
Adds a `calendar` area to the local store. It syncs calendars, participant
identities, events and tasks as masters, through CalendarEvent/changes,
and answers Calendar/get, CalendarEvent/query, /get and /set from SQLite
when the server can't be reached.

- A range query gets series in both client modes, with their overrides,
  and the calendar store expands them as it does for an older server.
- The synthetic-id probe gets the verdict from the last sync, so a client
  started offline behaves the same once the server is back.
- Create, edit, delete and RSVP queue in `calendar_pending`, one merged
  row per event, and go out through `calendar.flush` or the next sync.
  A create finds itself by uid after a lost response.
- A write the server refuses, or one on a value the server changed, goes
  to `calendar.conflicts`.
- `lib/store/calendar/reminders.ts` lists upcoming alerts with keys that
  don't depend on synthetic ids.
2026-11-03 05:42:01 +01:00
Linus Rath fdeb7963ba refactor: read event durations without loading the calendar UI 2026-11-03 05:41:35 +01:00
Linus Rath a4451dc900 feat: add results from the local store to online searches 2026-11-03 05:31:35 +01:00
Linus Rath fe5b5d5746 feat: search mail while offline, with compound words, prefixes and highlights 2026-11-03 05:31:34 +01:00
Linus Rath eb8db7f27d feat: keep offline changes, sends and conflicts in the local store's outbox 2026-11-03 05:30:39 +01:00
Linus Rath 51ccac02ca feat: answer every mail read from the local store, including header forms, identities and quotas, and keep attachments 2026-11-03 05:30:09 +01:00
Linus Rath 2bb0797996 feat: archive, mark read or reply right from a new mail notification
A notification for one message gets buttons where the browser has room
for them, and a click on its body works as before. The service worker
has no login, so an open tab takes the action first, through the store
actions. With no tab open a new same-origin route, /api/push/action,
does Mark read and Archive with the context cookie. When the cookie is
gone, the token has expired or the login can't be told apart, the
message opens in the app instead. Reply opens the composer through a
`?reply=1` link, with any text typed on the notification.

- `public/sw.js`: buttons, asking open tabs, the route, `sync` handling
- `app/api/push/action`: same-origin, checks the login and the account,
  honours the admin's Archive gate, files by the mirrored archive mode
- `lib/pwa-notify`: the page's side, the mirror for the worker, and
  `requestOutboxSync`, which the offline queue calls
- Background Sync, thin version: a `sync` event tells open tabs to flush

Two push-preview-route assertions change: they match the whole answer
for a named message, which now also carries `login`, what the action
route checks so a stale notification can't act on whoever holds the
cookie slot now.
2026-11-03 05:26:43 +01:00
Linus Rath bad156eaed refactor: let each part of the local store expose its own calls to the page 2026-11-03 04:59:39 +01:00
Linus Rath 6da9a6a6f5 feat: add an encrypted local mail store that syncs in a worker 2026-11-03 04:57:20 +01:00
Linus Rath 1dfe09d321 fix: keep private files and secrets out of standalone builds 2026-11-03 04:14:47 +01:00
Linus Rath af32c846a2 fix: let plugins read the signature auto-insert toggles they may change 2026-11-03 02:46:51 +01:00
Linus Rath 34e016f1c0 test: stop the server-state guard counting empty as-const tuples as state 2026-11-03 02:35:42 +01:00
Linus Rath 5c0ffd7aff fix: let plugins switch the signature auto-insert toggles and refuse the proofreading settings 2026-11-03 02:35:01 +01:00
Linus Rath e2bc222f49 fix: load the assistant's calendar and settings calls on first use so plugin views don't load the calendar store 2026-11-03 01:54:32 +01:00
Linus Rath ddbff7aeb7 Merge the assistant's Files (package D)
# Conflicts:
#	components/email/__tests__/composer-plugin-edit.test.tsx
#	lib/__tests__/plugin-projection.test.ts
2026-11-03 01:48:30 +01:00
Linus Rath 553f7ee883 fix: start a plugin's api calls at once again, not a tick later 2026-11-03 01:38:42 +01:00
Linus Rath e9e07ddb64 Merge main into feat/assistant-everywhere 2026-11-03 01:35:57 +01:00
Linus Rath cf512f0a4e fix: check attachments, shared folders and limits in the assistant's Files calls
Fixes from the review of api.files:
- `files.attach` to a new draft runs `onBeforeAttachmentUpload` for each
  file and the server's total size limit before `compose.open`, which
  runs neither; refused files come back in `refused`
- a write is refused when a folder above the target is missing from the
  listing, instead of counting it as not shared
- `onFileMove` reports the item as it was, from where it was, like the
  Files app's own move
- writes are counted against the 20-a-minute limit before the drive is
  listed, so refused writes count and cost no listing
- reads stop the download past 20 MB (`fetchBlobArrayBuffer` with a
  limit) and answer `too_big` when the listed size was wrong
- when the server refuses the new content of a replaced file, the old
  version's copy is taken out of the trash again

The two getText assertions on `fetchBlobArrayBuffer` now expect the
account (undefined for the own drive) and the byte limit, since the
call passes both.
2026-11-03 01:31:05 +01:00
Linus Rath d7cb895ebf Merge branch 'feat/assistant-everywhere' into feat/assistant-d
# Conflicts:
#	lib/__tests__/plugin-permission-coverage.test.ts
2026-11-03 01:19:42 +01:00
Linus Rath 55d9c81210 Merge more actions for the assistant (package C) 2026-11-03 01:13:47 +01:00
Linus Rath 6c3e68ce4e Merge branch 'feat/assistant-everywhere' into feat/assistant-d 2026-11-03 01:01:29 +01:00
Linus Rath 989cfc64a4 fix: offer a plugin the invitation and settings calls only once it was granted their permission 2026-11-03 00:55:50 +01:00
Linus Rath 37aaf05d6d fix: spend a plugin's undo token only once its undo went through, so a failed undo can be tried again 2026-11-03 00:51:30 +01:00
Linus Rath 169100eb12 fix: take the setting names app.getSettings gives, and check hidden, gated and restricted settings on undo too 2026-11-03 00:47:35 +01:00
Linus Rath 8add856aac fix: let plugins only switch off sender favicons and avatars in Junk
Both make the server fetch the sender's domain. The test that every
default value is allowed now also names senderFavicons (on by default,
which a plugin may no longer set); the rest of it is as it was.
2026-11-03 00:46:08 +01:00
Linus Rath d973e9062a fix: give a photo's date only from when it was taken, not when the file changed
exif.test.ts: the case "falls back to the file date" now expects no
date, since IFD0's DateTime is the file's change date, not the date
the photo was taken.
2026-11-03 00:45:56 +01:00
Linus Rath 22a783efe0 fix: check an invitation's sender also when it claims one of the user's own addresses 2026-11-03 00:43:16 +01:00
Linus Rath f7a7828bfa fix: check an image attachment's size before decoding it, and decode on older browsers
The host reads the size a PNG, GIF, JPEG, WebP or BMP declares in its
header first: over 50 megapixels it answers too_big, trackers and tiny
images are never decoded, a format whose header it can't read is
unsupported, and large images are decoded at the target width. A browser
that throws a TypeError on createImageBitmap's options decodes without
them.

The tests' stand-in pictures are now PNG headers instead of JSON (the
code reads the header), so image-shrink.test.ts and
plugin-host-attachment-image.test.ts build them that way, and the first
shrink test expects the resize options on createImageBitmap.
2026-11-03 00:42:53 +01:00
Linus Rath 6ef1397a53 fix: refuse to switch the out-of-office on for a plugin while it forwards mail or that can't be read 2026-11-03 00:42:30 +01:00
Linus Rath f8de35dd85 feat: let plugins count a conversation's messages, and read only the newest 50
api.email.getThread fetches only the newest 50 messages of a thread
(Thread/get, then Email/get for those ids) instead of all of them with
their bodies. api.email.getThreadCount answers the number of messages
from Thread/get, with its own rate limit.

plugin-host-thread.test.ts now expects getThreadEmails to be called with
{ newest: 50 }, the new option that limits what is fetched.
2026-11-03 00:38:07 +01:00
Linus Rath d68970126e feat: tell plugins when the open message's attachments can't be read from this login 2026-11-03 00:33:59 +01:00