Fixes from the review of the Files work (1cf9944b..a9a25b94).
- A file shared on its own, or one below a folder its owner took back,
could not be reached. Once Stalwart 1.0's placeholder folders were
left out it became a root of "Shared with me", which listed folders
only, and search opened it on the drive root. 0.16 lists such files
as roots too. Each owner's files shared on their own are now listed
on one page under "Shared with me" ("Files from <name>"), where they
open, preview and download like any other file and nothing can be
added; two of the same name get a number. Search opens a shared hit
where it is listed, and links into a shared drive name the node by id
(`/files?shared=<owner>:<id>`), so a shared folder or page also
survives a reload. A Files page that is already open takes the link
from search directly, since Next keeps it mounted when only the query
changes.
- A copy that fails part-way destroys only the nodes it made, the
deepest first and without `onDestroyRemoveChildren`, one call per
level because 0.16 refuses a folder destroyed in the same call as
its children. It used to destroy the new folder with everything
below it, so whatever someone put there during a long copy was
deleted for good. A folder holding such nodes now stays, and the
error says part of the copy was left in place.
- The document editor opens read-only on 0.16 for a sharee without the
rename right. 0.16.6 to 0.16.23 refuse any change a sharee makes
without Modify, so every save of such a document failed with a 500.
1.0, told apart by its `maxFileNodeDepth`, still asks for
`mayModifyContent` alone.
- Ctrl+C with only symlinks selected empties the clipboard instead of
keeping an earlier cut for the next paste.
- A paste reads each account's file tree once, not once per pasted
item for the listing and 1.0's depth check.
- The Stalwart bundle's README says that on older 0.16 releases and
the 1.0 pre-releases a failed update also drops the cached copy, and
the next restart or reload of the Applications downloads it again,
not only a later `UpdateApps`. The static README says a restart
applies the CORS setting on the current 1.0 pre-releases, where
`ReloadSettings` fails with the stock spam-filter rules.
Four smaller calendar fixes found in review of the Stalwart 1.0 work.
- A range load that failed left `dateRange` on the range loaded before,
so a push or a pull-to-refresh reloaded that one instead of the range
on screen, and any action that reset `error` hid the "may be out of
date" notice. The store now takes the range asked for and keeps the
failure in `eventsLoadError`, which only a successful range load
clears.
- The agenda plugin's expanded query had no sort, so its limit could
keep later occurrences instead of the next ones. It now sorts by
start, looks less far ahead when the server refuses the range as too
long (`expandDurationTooLarge`, an administrator's shorter
`maxExpandedQueryDuration`) instead of answering 502, and sends its
gets several per request.
- With a limit (search, contact activity) the chunks of a long range
went out one request each. Up to four now share a request, and what
the later ones return past the limit is dropped. Global search passes
its AbortSignal down, so a replaced search stops between requests.
- A search of several words sends its date bounds as conditions of a
flat AND. The client did not see them as a range: it neither expanded
recurrences nor turned the UTC bounds into the request's wall clock,
so such a search found series instead of occurrences, over a range
shifted by the zone's offset. The bounds are now lifted out of the
AND, converted and chunked, and each chunk keeps one flat AND of
single conditions - the shape 1.0 expands; it refuses an AND holding
a condition with both bounds.
Copying across accounts went through Blob/copy, which failed with
`unknownMethod` on Stalwart 0.16.10+ and 1.0 because
`urn:ietf:params:jmap:blob` was not in `using`. Even with it, the copy
would have been reserved to the target account, which only its members
may use, so copying into another user's shared drive stayed refused.
Copies now point at the source's blobId: Stalwart resolves a FileNode
blob from another account for anyone who may read that node, in both
directions and on every supported version.
A folder copy lays out the whole subtree before the first create. If a
create fails part-way, the half-made copy is destroyed with its
children and the server's error is reported, instead of leaving part
of the tree behind. Files created over WebDAV inside a copied folder
are no longer copied under their percent-encoded names (#869).
New files are created without `size`: every server takes it from the
blob, and 1.0 refuses a size that differs from the blob's.
Stalwart 1.0 treats a FileNode's `modified` as the client's: a content
update leaves it alone unless the patch asks for it. Saves from the
document editor therefore kept the old date in the file list, and the
editor's conflict check (409 with COOLStatusCode 1010) never noticed a
change made by another JMAP client. PutFile now sends `modified: null`,
which has the server stamp the save. 1.0 echoes the new time in
`updated`; 0.16.6+ answer `null` as before and the batched FileNode/get
reads it.
The editor also could not open a file in someone else's shared drive,
on any version: the Files app sent the listing's "<owner>:<id>" id for
the user's own account, and the launch failed with a 404. It now names
the owner's account and the node's own id, so owner and sharee join
the same editor session. A save uploads into the user's own account
and then points the owner's node at that blob, because Stalwart only
lets members of an account use a blob uploaded into it. The access
token carries that upload account; tokens minted before it upload into
the document's account as before.
The editor now also uses the cookie slot of the login the Pro shell is
browsing, not always the active one.
Stalwart 1.0 reads a text filter value that starts and ends with a
quote as one exact phrase, and a value ending in `*` as one prefix over
the whole string, not split into words and capped at 16 bytes. Every
search path sent the typed input as one `text` value, so on 1.0
`"annual report" 2024 "draft"` became a single four-word phrase and
`project budg*` a prefix containing a space: both found nothing.
`"Q3 planning" budget` lost its phrase on every version.
`splitSearchText` (`lib/jmap/search-utils.ts`) now sends each quoted
phrase and each `word*` as a condition of its own, ANDed with the rest.
The remaining words stay one condition, so a query without quotes or
stars goes out as before. Outer single quotes are stripped from that
part, since 1.0 reads them as a phrase too. The split applies to:
- the search bar, tag views, cross-account and all-folders search;
- advanced search, including the from/to/subject/body fields;
- contact search;
- global search for mail, contacts and calendar.
`inMailbox` and the calendar bounds sit next to the text conditions in
one AND, with one key per condition: Stalwart rejects an operator that
also carries property keys, and 1.0 allows only one AND group when it
expands recurrences.
SearchSnippet/get switches the whole request to exact matching once a
term is quoted, which marks nothing for a phrase ANDed with other
words, so the snippet filter unquotes phrases.
On 1.0 and on external search backends each phrase is now a real
phrase. The 0.16 built-in index reads it as an AND of its words,
without stemming. The dev mock and the demo client now evaluate AND
filters and quoted phrases, so they keep working with the new shapes.
Stalwart 1.0 counts every EmailSubmission record of an account against
`maxSubmissions` (500 by default), whatever its status or age, and its
own purge of old records never runs. Bulwark never destroyed a
submission, so after about 500 JMAP sends an account could not send at
all: every send, read receipt and reschedule failed with a raw "There
are too many email submissions" error, and each failed attempt left one
more copy of the message in Drafts.
Bulwark now removes the records that can no longer be cancelled.
Destroying a record never stops its delivery, but a pending one could
no longer be cancelled, so pending records are never touched:
- an immediate send (no hold, no undo window) destroys its own record
right after the send, riding along with the old-draft cleanup; only
where `urn:stalwart:jmap` is advertised, and never decided by the
created `undoStatus`, which 0.16.10-0.16.16 report as final for held
sends
- a throttled background prune destroys canceled records and final
records whose `sendAt` has passed, going by `EmailSubmission/get`, in
every submission account; it runs at startup, after sends, after
undo and on EmailSubmission state changes
- a confirmed cancel (undo, cancel, reschedule) destroys the canceled
record in a request of its own
- an `overQuota` on a submission create prunes at once, on Stalwart
walks the change log for records older than the query's three-day
window, retries once, and otherwise throws `SubmissionQuotaError`,
shown as a translated explanation in the composer, quick reply, read
receipts and the scheduled-send actions
Failed sends now clean up after themselves and say so:
- a failed send removes the Drafts copy it created and keeps the
user's draft; the raw send paths remove their import the same way
- a read receipt that did not go out is taken back out of Sent, a
method-level error counts as a failure, `$MDNSent` is only set after
a sent receipt, and the automatic receipt no longer retries on every
refresh of the open message
- quick-reply failures, "Send now", "Reschedule" and a failed undo are
shown instead of only logged
- the scheduled list reads every page of `EmailSubmission/query`; it
stopped after the first because `total` needs `calculateTotal`
Stalwart 1.0 refuses a CalendarEvent/get that asks for
recurrenceOverrides together with utcStart or utcEnd, and every calendar
read did. The grid, tasks, reminders, search, contact activity and the
agenda plugin came back empty with no error, and creating an event or
task threw after the server had created it, so a retry made a
duplicate.
- read overrides with a second get in the same request, only where
they are needed
- treat an error answer to a get or query as a failure; the calendar
keeps the last events and says it could not load them, with a retry
- never throw after the server created an event or task
- let the server expand series for /api/calendar-agenda
A PatchObject key is a JSON Pointer (RFC 8620 section 5.3), so a slash
inside the member name it points at has to be escaped as `~1`. Keyword
patches were built as `keywords/${keyword}` verbatim, which is fine for
every flat keyword but wrong for a nested tag: `$label:work/clients`
became the pointer `keywords/$label:work/clients`, asking for the
`clients` member of the `$label:work` keyword rather than for the keyword
itself. The patch is not a boolean at that path, so the server rejects or
ignores it and the tag never lands.
Route the keyword patch sites - setKeyword, removeKeyword, migrateKeyword
and the category-tab batch patch - through `keywordPointer`, which escapes
per RFC 6901 the way the calendar store already does for participant ids.
Flat keywords are untouched, so `$seen`/`$flagged` and friends keep the
exact pointers they had. The dev JMAP mock reads the escaping back, or
nested tags would behave differently against the mock than against a real
server.
migrateKeyword also took any answer as success: a method-level error on
the query read as "no message carries this keyword", one on the write was
ignored outright, and a per-message refusal was invisible - so a rename
that moved nothing still reported a clean migration. It now answers with
{ migrated, refused }, because the work is batched and a failure partway
through cannot undo the batches already applied. A whole batch the server
refuses stops the rest, which it would refuse too; a single message it
refuses does not, the mail behind it being unaffected. A query error
throws, nothing having been written yet, and so does a run that migrated
nothing at all - the case where the caller must not rename a tag over
messages that all kept the old keyword.
The walk over tagged mail was not paginating safely either. It stopped at
the first page shorter than the limit it asked for, but RFC 8620 section
5.5 lets a server clamp that limit, so a clamping server left most of the
mail unmigrated. Positions are also only comparable within one queryState,
which the walk ignored: mail arriving mid-walk shifts every later page. It
now restarts when the state changes under it, and gives up - before any
write, so the caller can simply repeat - once a mailbox has moved under it
three times, rather than mixing ids from states that no longer agree. A
page carrying no id it has not already seen ends the walk, which keeps it
bounded against a server that answers every position with the same page.
contactUsing(), calendarUsing(), and fileUsing() checked the base
"principals" capability but then declared the separate
"principals:owner" sub-capability (RFC 9670) in the request's "using"
array. RFC 8620 section 3.3 requires a server to reject the whole
request if it declares a capability the server does not advertise, so
a server that supports base principals without the owner
sub-capability rejects every one of these requests with
unknownCapability.
Fixed to check the specific sub-capability being declared instead of
assuming it from the broader one. The calendar agenda route had the
same mismatch and is fixed the same way. Note that RFC 9670 section
1.5.2 places principals:owner in accountCapabilities rather than the
session capabilities, so on compliant servers the sub-capability is
simply never declared, which is the intent. Nothing is lost by
omitting it.
The file share test asserted the old behavior by expecting the owner
capability to be declared against a server that advertises only base
principals. It now asserts the capability is omitted in that case and
declared only when the server advertises it. The same pair of
assertions is added for the contacts, calendars, and agenda route
request builders, so each site this commit touches is covered against
regression.
If you have a large number of tags, getTagCounts would not be able to get the
unread counts because it did not respect maxCallsInRequest, even though the
value was actually read out, it was just ignored. There are also other places
where the limits were not respected.
Batching is now generalized in a helper that also takes maxObjectsInSet, which
also was ignored, into account and is applied to all functions.
In addition, the dev mock now also advertises and enforces the limits, so these
issues can get picked up during development.
Possible closes#699
Possibly closes#399
Linus Rath
Linus Rath
Paulhenry Saux
KazNIISA IT
zisisvakras
Eduard Polonskyi
Mathy Vanvoorden