From d6cf1ef9872cd0db0f51e4881237bac383e45f61 Mon Sep 17 00:00:00 2001 From: Guilhem Date: Tue, 21 Jul 2026 19:06:21 +0200 Subject: [PATCH] feat: attach text files to chat messages, read on demand via file tools (#10215) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: attach text files to chat messages, read on demand via file tools * fix: resolve name collisions and sync message files with the transcript * refactor: render message file chips with the shared context badge * fix: suffix same-name attachments, sync registry on compaction, bound file bytes * feat: carry message files across compaction, drop them from the roster * refactor: merge context, dom and file badges into one wrapping row * fix: dedupe identical attachments and make badge list keys collision-proof * fix: name carried files inside the collapsed summary instead of badges * fix: serialize registry reconciliation and correct the attachment budget * fix: reserve pending bytes so overlapping reads honor the attachment budget * fix: share attachment byte budget across concurrent composers The bottom composer and the edit box are both mounted while editing an earlier message, but each enforced MAX_CONVERSATION_FILE_BYTES against only its own staged files plus the transcript. Two attaches near the cap could each pass independently and overflow the persisted transcript. Each composer now publishes its staged bytes (committed attachments + in-flight reads) to the manager, keyed per instance, and the attach-time budget subtracts every other live composer's stage. A message an open composer is editing is skipped from the transcript sum since that composer's stage stands in for it. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: charge an edited message at its persisted size until the edit commits An edit is not committed until send, so the edited message's persisted attachments return if the edit is cancelled. Substituting only the edit box's (possibly emptied) stage let the always-mounted bottom composer claim headroom that vanishes on cancel: remove the near-limit files in the editor, fill the bottom draft, cancel, and the persisted transcript overflows MAX_CONVERSATION_FILE_BYTES. attachmentBytesExcluding now charges a message another composer is editing at max(persisted size, editor stage), so freed space only becomes available once the edit actually commits. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: preserve a message file's exact name when a session row clashes on rebuild syncMessageScoped rebuilds message-scoped rows from the transcript through collision-suffixing addFiles. Session rows load first (on restore), so one holding a wanted name pushed the rebuilt message row to a "(2)" suffix while the persisted prompt still referenced the bare name — get() then resolved the reference to the session asset and the model read the wrong content. Free the name from the conflicting session row before the rebuild so the message row reclaims its exact reference. The rename is in-memory only: it is deterministic and re-applied on every load, and the session roster is regenerated live each send, so the session asset stays addressable under the suffix without a persisted-record update. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: reserve resent files across the edit-resend gap The edit box unmounts (dropping its staged-byte entry) the instant the user submits an edit, but restartGeneration then awaits registry sync and beforeSend before the optimistic bubble lands in the transcript. During that gap the resent files were reserved nowhere, so the always-mounted bottom composer could attach into the temporary headroom and the resend would then push the persisted transcript past MAX_CONVERSATION_FILE_BYTES. restartGeneration now reserves the resent files' bytes in shared manager state before the transcript slice; sendRequest releases the reservation once it installs the bubble (or restores the files to the composer on a pre-install bail). The reservation bridges the gap so the budget stays honored throughout. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: re-target an in-flight index when a session row is renamed on rebuild #freeNameForMessageRow renames a restored session row so a same-named message attachment can reclaim its exact name. But the row's #indexFile, started under the old name during restore, stamps via #patchFile(oldName, file) — after the rename that no longer matches, leaving the row stuck 'indexing' so read_file rejects it and search_files excludes it. Re-kick #indexFile under the new name when the renamed row is still indexing; the stale completion then no-ops (its name is gone). Co-Authored-By: Claude Opus 4.8 (1M context) * fix: release resend reservation on local-command sends; surface compaction-orphaned files Two follow-ups to the message-attachment work: - A resend edited to /clear or /compact runs the local-command path and returns before installing a bubble, so the #RESEND_KEY reservation set by restartGeneration was never released and its bytes stayed charged, blocking later attachments. Release it on every sendRequest path that exits before install (via #releaseResendReservation). - Drop-oldest compaction (summary fallback) removes API messages without a summary, so a folded message's `## ATTACHED FILES` reference no longer reaches the model even though the file stays readable. The roster omits message-scoped files, so the model loses awareness of them. orphanedMessageFileNames() finds message files whose only referencing message went negative-index, and the roster now advertises them (summary compaction already carries its own). Co-Authored-By: Claude Opus 4.8 (1M context) * fix: key the resend reservation per send so unrelated sends can't release it The resend reservation used a single shared key, so a normal or concurrent sendRequest released it at its own install/early-return even though it didn't own it — dropping an in-flight resend's reservation and letting attachments staged before the resend bubble lands under-count against the byte cap. restartGeneration now mints a per-resend token, reserves under it, and threads it through sendRequest as resendReservationKey; releases act only on that key. A send with no token (every normal send) releases nothing. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: reserve the whole wanted-name set when freeing session rows on rebuild Freeing a session row for a message-scoped rebuild suffixed it against current rows only. With the transcript referencing both notes (2).md and notes.md and a session row named notes.md, freeing notes.md renamed it onto notes (2).md — also a wanted reference — so that message row cascaded to notes (3).md while its persisted reference stayed notes (2).md, and read_file returned the session file. #uniqueName now accepts a reserved set; the rebuild frees each session clash clear of the entire wanted-name set, so every message row reclaims its exact reference regardless of collision order. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: orphan summary-carried files when drop-oldest removes the summary orphanedMessageFileNames classified summary rows as always-live, but a summary carries its folded files' reference on its own API message. When summary compaction succeeds and a later summarization fails, drop-oldest can remove that API message, yet the files stayed off the roster — so the model lost their reference even though they remained readable. The summary display row now tracks its API index (slot 0 at creation, re-based by drop-oldest); a negative index reads as "counterpart gone" and its files move to the roster, mirroring user-message orphans. The index is used only for orphan detection, never as a restart target. Co-Authored-By: Claude Opus 4.8 (1M context) * fix: reserve outgoing file bytes for normal and queued sends too The resend reservation covered edit/retry, but a normal or queued send has the same gap: the composer (or queue) clears its files the instant sendRequest is called, dropping the staged-byte entry, while sendRequest then awaits regrantLocked()/refreshFolders() before the bubble lands. With a locked or slow linked folder the composer stays enabled, so a fresh drop can spend the same headroom and overflow the 5 MB cap once the first bubble installs. Generalize the reservation: sendRequest mints a per-send token and reserves the outgoing files' bytes just before attachment upkeep (reusing restartGeneration's token when present), and releases it on install or any pre-install exit. Co-Authored-By: Claude Opus 4.8 (1M context) * refactor: join attachment layers on a stable content-hash id Sixteen review rounds kept finding the same bug family: a message file's identity was its display name, joined by hand across the registry, the transcript, the prompt reference, and the render keys — every same-name collision or interleaved rebuild made two of those copies disagree. Give each message attachment a deterministic id, attachedTextFileId(name, content) — a synchronous pure-JS hash (works on plain-HTTP deployments) whose exact value is pinned by test, since persisted transcripts reference it. The prompt and roster list the id, the file tools resolve id-first (bare names remain a fallback for legacy chats and session links), and pre-id transcripts hydrate on load by recomputing the same hash — no migration state. Names become display-only and may collide freely, which deletes the machinery that defended them: the suffix-readback registration loop, session-row renaming on rebuild (#freeNameForMessageRow, reserved-set #uniqueName), and the reconciler's serialization guards (#syncSeq/#syncChain) — syncMessageScoped now compares ids instead of awaiting blob text, so it is synchronous and cannot interleave. A same-name clash within one draft gets a composer-local courtesy rename before the id is minted. Co-Authored-By: Claude Fable 5 * fix: resolve bare names to session rows and scope id searches to one row A bare name is the roster's namespace: session links are advertised by filename and have no other handle, so a same-named message attachment (which is addressed by id) must not shadow them. get() now resolves session rows first, keeping the message-row name lookup only as the fallback for transcripts persisted before ids existed. search_files restricted an id reference by mapping it back to the display name and letting the worker filter on it — same-named files were then searched together under one label. The tool now passes the resolved row itself, so an id-scoped search can only ever hit its own file. Co-Authored-By: Claude Fable 5 * fix: label search hits with resolvable ids, normalize attach batches at commit An unscoped search_files reported hits by display name only. Names may collide, so a hit could not be mapped back to the row that produced it — a follow-up read_file on the bare name could return a different same-named file. Rows carrying an id are now labeled `name (file id: …)` in hit lines, so every hit names the reference that resolves to exactly that row. addTextFiles normalized (deduped, courtesy-renamed) each file against a snapshot taken during its read loop. Attach batches overlap, so a file committed by another batch between reads escaped both checks — duplicate or same-named unsuffixed entries in one message. Normalization now runs in the single synchronous commit step against the live list (foldIntoDraft), where nothing can interleave. Co-Authored-By: Claude Fable 5 * fix: dedupe renamed re-drops in foldIntoDraft, truncate queued file chips "Same file dropped twice" means same original (name, content), but a courtesy rename erases the original name — an identical re-drop then missed the duplicate check and landed as a further-suffixed copy. The dedupe now also matches entries whose suffix-stripped base name equals the read's name. Queued file chip labels get min-w-0 so long filenames truncate inside max-w-36 instead of overflowing. Co-Authored-By: Claude Fable 5 * fix: address cubic review — line counts, chip clicks, reference robustness Five fixes from the cubic pass: - The prompt advertised split('\n').length lines, one more than read_file reports for newline-terminated files — textLineCount now matches the tool's numbering (0 for empty, no phantom trailing line). - Clicking a sent message's badge opened edit mode (the wrapper's click-to-edit), unmounting the preview popover as it opened; the badge row now keeps clicks to itself. - resolve() accepts the composite label rosters and search hits print (`name (file id: x)`) — models echo references verbatim, so the printed form must resolve. - fileToAttachedTextFile enforces MAX_TEXT_FILE_BYTES itself (raw size + decoded byte length), so no ingestion path can persist an oversized attachment past the composer's pre-check. - Duplicate detection after a courtesy rename now uses an explicit sourceName instead of inferring provenance from the display name — a user's real `report (2).md` is not a rename of `report.md`. Co-Authored-By: Claude Fable 5 * fix: exact names win over label parsing, commit recheck uses decoded bytes resolve() parsed any `name (file id: x)`-shaped reference as a printed label, so a session file literally named that way became unreachable by its exact name (the dead-id fallback resolved the base name instead). Exact id and exact-name lookups now run before label interpretation. Attachment admission and the pending reservation use raw File.size, but the committed charge is the decoded UTF-8 length — malformed input decodes each invalid byte to a 3-byte replacement character, so a file passing the 8KB text sniff could inflate past the conversation cap. The synchronous commit step now re-checks the live budget against decoded sizes (admitWithinByteBudget) and drops what no longer fits, with the budget toast. Co-Authored-By: Claude Fable 5 * fix: normalize files folded into the queued message Repeated submissions during a stream aggregate into one queued message, but their files were concatenated raw: an identical re-attach duplicated its chip and ate a slot (possibly displacing a distinct file at the eight-file cap), and a same-name clash skipped the courtesy rename. The queue now folds new files through the same commit normalization as the composer — the queued entry is a message draft like any other. Co-Authored-By: Claude Fable 5 * style: compaction boundary label uses text-normal text-2xs * fix: fold provenance survives pass-through, dequeued files fold into the draft foldIntoDraft recorded sourceName only for renames it performed itself, so a file already courtesy-renamed by the composer lost its provenance when folded into the queue — a later re-attach of the original escaped dedupe. Folds now compose: the original source name rides through every fold, and dedupe matches on it. dequeueMessage restored queued files into a possibly-populated composer by raw concatenation; prependText now folds them like every other draft aggregation (dedupe, courtesy rename) before applying the cap. Co-Authored-By: Claude Fable 5 * refactor: one MessageDraft owns the lanes that ship with a send Review rounds kept finding the same P2 shape: an aggregation point where files join a draft (composer commit, queue append, dequeue restore) that forgot one of the draft rules — fold dedupe, courtesy rename, slot caps, byte admission, lanes moving together. The rules existed only as convention re-implemented per site. MessageDraft owns them once: text, pastes, images, and text files live on one object with addFiles (fold + optional decoded-byte admission + cap), addImages (cap), prepend (restore-merge), replaceIfEmpty (occupied-guard restore), and take (all lanes leave together). The composer holds a draft instead of four state vars, and the queue is a draft behind the existing queuedMessage/queuedImages/queuedFiles accessors — an aggregation point can no longer skip a rule, because there is no raw array to concatenate into. Deliberately not moved: @context and DOM picks (ContextManager owns their lifecycle), the conversation byte budget's cross-composer ledger (store-side follow-up), and sendRequest's options shape (it decomposes immediately and is pinned by the manager test suite). Co-Authored-By: Claude Fable 5 * docs: correct the drop-routing comment, condense the budget doc * fix: address cubic review — name sanitization, drop hold, merge restores Four fixes from the cubic pass: - Attachment display names render into model-facing prompt blocks, and OS filenames may legally contain control characters — sanitizeAttachmentName strips them at attach and again at every prompt-render site (legacy names predate the attach-time pass), so a crafted name cannot inject prompt lines. - Drop routing awaits handle/entry resolution before it can call addTextFiles; a send during that window landed the dropped files on the next message. The drop handler now holds sending (holdSendForIngestion, taken before the first await) until routing completes. - Restoring a taken queue after a failed auto-send replaced the queued draft wholesale, silently losing a follow-up queued during the preflight. Both #restoreQueue and the unmounted-input requeue now merge via draft.prepend — the taken entry's text lands above the newer follow-up. - restartGeneration validated the API restart index only after reserving the resend bytes and truncating the transcript, so a stale index threw with the reservation leaked and the display transcript half-mutated. The index is resolved and validated before anything is touched. Co-Authored-By: Claude Fable 5 * fix: restored drafts keep chronological priority, store names stay resolvable A failed auto-send's restore folded the taken (older) draft's attachments AFTER a follow-up queued during preflight, so at the slot caps the older attachments silently dropped despite the text landing first — and only one entry's pinned context survived. prepend() now puts the restored lanes ahead (the cap drops the newest additions) and #restoreQueue unions both pinned contexts by identity. Session filenames were sanitized only at prompt render, so an id-less file whose stored name carries control characters was advertised under a name that resolve() could not match. Names are now sanitized at every store row-creation site (attach, folder expansion, refresh, and persisted-row restore for pre-sanitization records), making the advertised name the stored name everywhere — render-site sanitization remains as defense in depth. Co-Authored-By: Claude Fable 5 * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 4c08634af953db5c1125b1fb03f5af211fe21db3 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore: repin ee-repo-ref to main's eb3690a34b (aaa6cb89 needs the unmerged AmqpTrigger OSS companion) * Revert "chore: repin ee-repo-ref to main's eb3690a34b (aaa6cb89 needs the unmerged AmqpTrigger OSS companion)" This reverts commit a782e6f1c5a4c054fc660ce1f432020e1019aeb7. * refactor: one name seam per scope — claim on the store, sanitize in the fold Name rules (sanitize, uniquify, dedupe, resolve) lived as convention at every creation site: four sites carried verbatim copies, two skipped uniquify (restore, file placeholders — legacy names could collapse to one display name), and the dedupe check compared the raw name against sanitized stored names, so re-linking a control-char file added a spurious copy. Store side: #claimName(raw) = sanitize + uniquify is now the only way a session row gets its display name; addFiles derives the sanitized name once at the top of the loop, so dedupe, uniqueness, and the stored row all see the same string, while relPath and folder keep the raw on-disk keys they must match. Draft side: foldIntoDraft sanitizes its reads itself instead of assuming the reader did, making the fold self-contained. Folder names — raw grouping keys by design — are sanitized at their model-facing render sites (roster folder lines, not-found listing), mirroring rosterLine. Co-Authored-By: Claude Fable 5 * fix: a display-name collision is not a duplicate Sanitizing names before the dedupe compare made two DISTINCT files whose raw names sanitize identically look like re-links — the second was silently discarded instead of claiming a suffixed name like the restore path does. Dedupe now means "the same file re-linked": matching stats (size + mtime) plus a matching name, pre-suffix sourceName, or raw path. #claimName records the pre-suffix name when uniquifying renamed a row, so re-linking a suffixed file's original still dedupes. A same-named file with different stats links as its own row rather than being silently swallowed. Co-Authored-By: Claude Fable 5 * fix: the re-link identity is the raw name, persisted with the row Dedupe still compared sanitized display names, so two distinct raw names that sanitize identically collapsed whenever their stats also matched — and the provenance recorded for suffixed rows lived only in memory, so after a reload re-linking the original behind a suffixed row stacked another copy. sourceName now records the RAW pre-sanitization name on every session row (display names lose information twice — sanitize, then suffix), rides the persisted record, and is re-derived on restore. Dedupe matches stats plus raw identity (sourceName, or relPath for folder children) and never compares display names. #claimName returns to a pure name function; restore claims the display name from the persisted raw identity. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Ruben Fiszel Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: windmill-internal-app[bot] --- .../copilot/chat/AIChatDisplay.svelte | 79 ++- .../copilot/chat/AIChatInput.svelte | 376 +++++++++---- .../copilot/chat/AIChatManager.svelte.ts | 510 ++++++++++++++---- .../copilot/chat/AIChatManager.test.ts | 293 +++++++++- .../copilot/chat/AIChatMessage.svelte | 32 +- .../copilot/chat/CompactionBoundary.svelte | 12 +- .../copilot/chat/ContextElementBadge.svelte | 16 +- .../copilot/chat/ContextTextarea.svelte | 16 +- .../copilot/chat/HistoryManager.svelte.ts | 12 +- .../copilot/chat/QueuedMessageChip.svelte | 30 +- .../lib/components/copilot/chat/context.ts | 22 +- .../chat/files/AttachedFilesBar.svelte | 2 +- .../chat/files/attachedFiles.svelte.ts | 248 +++++++-- .../copilot/chat/files/attachedFiles.test.ts | 231 ++++++++ .../copilot/chat/files/attachedFilesDB.ts | 3 + .../copilot/chat/files/fileTools.test.ts | 66 ++- .../copilot/chat/files/fileTools.ts | 85 ++- .../copilot/chat/global/core.test.ts | 31 ++ .../components/copilot/chat/global/core.ts | 23 + .../copilot/chat/messageDraft.svelte.ts | 152 ++++++ .../copilot/chat/messageDraft.test.ts | 91 ++++ .../src/lib/components/copilot/chat/shared.ts | 18 +- .../copilot/chat/textFileUtils.test.ts | Bin 0 -> 6684 bytes .../components/copilot/chat/textFileUtils.ts | 208 +++++++ 24 files changed, 2212 insertions(+), 344 deletions(-) create mode 100644 frontend/src/lib/components/copilot/chat/messageDraft.svelte.ts create mode 100644 frontend/src/lib/components/copilot/chat/messageDraft.test.ts create mode 100644 frontend/src/lib/components/copilot/chat/textFileUtils.test.ts create mode 100644 frontend/src/lib/components/copilot/chat/textFileUtils.ts diff --git a/frontend/src/lib/components/copilot/chat/AIChatDisplay.svelte b/frontend/src/lib/components/copilot/chat/AIChatDisplay.svelte index aefa123713..a267e71e8c 100644 --- a/frontend/src/lib/components/copilot/chat/AIChatDisplay.svelte +++ b/frontend/src/lib/components/copilot/chat/AIChatDisplay.svelte @@ -39,6 +39,7 @@ import { getAiChatManager } from './aiChatManagerContext' import ChatTypingIndicator from './ChatTypingIndicator.svelte' import AIChatInput from './AIChatInput.svelte' + import AttachedFilesBar from './files/AttachedFilesBar.svelte' import QueuedMessageChip from './QueuedMessageChip.svelte' import JobsSegment from './JobsSegment.svelte' import { getModifierKey } from '$lib/utils' @@ -272,8 +273,8 @@ // File attachment is GLOBAL-mode only. const canAttachFiles = $derived(aiChatManager.mode === AIMode.GLOBAL && !disabled) - // Steers the OS file picker toward text + image formats (soft hint; images attach to - // the message, other files link as text context after a content sniff). + // Steers the OS file picker toward text + image formats (soft hint; both attach + // to the message — text files after a content sniff). const TEXT_FILE_ACCEPT = 'image/*,text/*,.txt,.csv,.tsv,.json,.jsonl,.ndjson,.md,.markdown,.log,.yaml,.yml,.toml,.ini,.cfg,.conf,.env,.xml,.html,.htm,.css,.js,.mjs,.cjs,.ts,.tsx,.jsx,.py,.rb,.rs,.go,.java,.kt,.c,.h,.cpp,.cc,.cs,.php,.sh,.bash,.zsh,.sql,.svelte,.vue,.dockerfile' let fileInputEl = $state(null) @@ -361,16 +362,30 @@ e.preventDefault() const dt = e.dataTransfer if (!dt) return - // Images attach to the message; other files link as text context. Images are - // reserved from dt.files BEFORE any await (a send mid-ingestion would land - // them on the next message), and dt.files is the only place a disk-less drag - // exists — a cross-tab image resolves every getAsFileSystemHandle() to null. + // Images and loose text files attach to the message; folders link as session + // assets. Images are reserved from dt.files BEFORE any await (a send + // mid-ingestion would land them on the next message), and dt.files is the + // only place a disk-less drag exists — a cross-tab image resolves every + // getAsFileSystemHandle() to null. const flatFiles = Array.from(dt.files ?? []) const topLevelImages = flatFiles.filter(isImageFile) const imageWork: Promise[] = [] if (topLevelImages.length > 0) { imageWork.push(aiChatInput?.addImages(topLevelImages) ?? Promise.resolve()) } + // Text-file routing must await handle/entry resolution before it can call + // addTextFiles — hold sending across that window (taken BEFORE the first + // await) or a send mid-resolution would land the drop on the next message. + const releaseSendHold = aiChatInput?.holdSendForIngestion() + try { + await routeDroppedTextAndFolders(dt, flatFiles) + } finally { + releaseSendHold?.() + } + await Promise.all(imageWork) + } + + async function routeDroppedTextAndFolders(dt: DataTransfer, flatFiles: File[]) { if (canUseFsAccess) { // getAsFileSystemHandle calls are kicked off synchronously inside this call. const handles = await handlesFromDataTransfer(dt) @@ -381,9 +396,9 @@ handles.length === 0 ? flatFiles : await Promise.all(handles.filter(isFileHandle).map((h) => h.getFile())) - // Files are always snapshotted (handle discarded). + // Loose text files attach to the message, like images. const textFiles = looseFiles.filter((f) => !isImageFile(f)) - if (textFiles.length > 0) await handleAddFiles(textFiles) + if (textFiles.length > 0) await aiChatInput?.addTextFiles(textFiles) // Folders link as a live handle. for (const h of handles.filter(isDirectoryHandle)) { await addDirHandle(h) @@ -395,19 +410,25 @@ // (no entry API), fall back to the flat dt.files. const entries = await readDroppedEntries(Array.from(dt.items ?? [])) const source: FileToAttach[] = entries.length > 0 ? entries : flatFiles - // Only top-level images attach to the message, and those were already - // reserved from dt.files before the walk — drop them here so they aren't - // re-reported as skipped non-text. Folder-nested images are deliberately - // NOT attached (the FSA path never extracts folder contents either); they - // ride the text ingestion and are summarized as skipped. - const textEntries = source.filter((entry) => { + // Top-level files attach to the message (images were already reserved + // from dt.files before the walk). Folder children keep riding the + // session store as a snapshot — including nested images, which are + // deliberately NOT attached (the FSA path never extracts folder + // contents either); they are summarized as skipped there. + const topLevelText: File[] = [] + const folderEntries: FileToAttach[] = [] + for (const entry of source) { const file = entry instanceof File ? entry : entry.file - const nested = !(entry instanceof File) && entry.path?.includes('/') - return !isImageFile(file) || !!nested - }) - if (textEntries.length > 0) await handleAddFiles(textEntries) + const nested = !(entry instanceof File) && !!entry.path?.includes('/') + if (nested) { + folderEntries.push(entry) + } else if (!isImageFile(file)) { + topLevelText.push(file) + } + } + if (folderEntries.length > 0) await handleAddFiles(folderEntries) + if (topLevelText.length > 0) await aiChatInput?.addTextFiles(topLevelText) } - await Promise.all(imageWork) } async function onFileInputChange(e: Event) { @@ -418,7 +439,7 @@ const textFiles = picked.filter((f) => !isImageFile(f)) // Reserved before the text work is awaited — see onPanelDrop. const imageWork = imageFiles.length > 0 ? aiChatInput?.addImages(imageFiles) : undefined - if (textFiles.length > 0) await handleAddFiles(textFiles) + if (textFiles.length > 0) await aiChatInput?.addTextFiles(textFiles) await imageWork } input.value = '' // allow re-selecting the same file @@ -754,10 +775,11 @@ the panel, or the Escape-to-stop focus check would wrongly reject them. --> {/if} - + mention deselects). Hence showContext={false} below. Session-scoped + assets (attached files/folders) render in the footer row instead. --> {#if inputPreface} {@render inputPreface()} {/if} @@ -863,12 +885,12 @@ the panel, or the Escape-to-stop focus check would wrongly reject them. -->

Attach files or link a folder

- Text files stay in your browser, and a folder is linked live from disk. - The assistant lists, searches, and reads them on demand, so their contents - are sent only when it reads one. + Files and images attach to your next message. Images are seen directly; + file contents stay in your browser and are read on demand.

- Images are sent with your next message, so the assistant can see them. + A linked folder is a session-wide resource: the assistant lists, searches, + and reads its files whenever it needs them.

{/snippet} @@ -876,7 +898,7 @@ the panel, or the Escape-to-stop focus check would wrongly reject them. --> {/snippet} + `accept` only steers the picker; the content sniff at attach is authoritative. --> {:else}
+ {#if aiChatManager.mode === AIMode.GLOBAL} + + {/if} {#if !hideModeSelector} {/if} diff --git a/frontend/src/lib/components/copilot/chat/AIChatInput.svelte b/frontend/src/lib/components/copilot/chat/AIChatInput.svelte index 427856d291..336246b368 100644 --- a/frontend/src/lib/components/copilot/chat/AIChatInput.svelte +++ b/frontend/src/lib/components/copilot/chat/AIChatInput.svelte @@ -2,10 +2,10 @@ import AppAvailableContextList from './AppAvailableContextList.svelte' import ContextElementBadge from './ContextElementBadge.svelte' import ContextTextarea from './ContextTextarea.svelte' - import AttachedFilesBar from './files/AttachedFilesBar.svelte' import autosize from '$lib/autosize' import { contextElementKey, + createAttachedFileContextElement, isSameContextElement, type AppDomSelectorElement, type ContextElement @@ -31,6 +31,16 @@ } from './imageUtils' import { modelSupportsVision } from '../modelConfig' import { tryGetCurrentModel } from '$lib/aiStore' + import { createLongHash } from '$lib/editorLangUtils' + import { + fileToAttachedTextFile, + MAX_ATTACHED_FILES, + MAX_CONVERSATION_FILE_BYTES, + MAX_TEXT_FILE_BYTES, + textByteLength, + type AttachedTextFile + } from './textFileUtils' + import { MessageDraft } from './messageDraft.svelte' import ExpandableImage, { isImageViewerOpen } from '$lib/components/common/image/ExpandableImage.svelte' @@ -46,6 +56,7 @@ initialInstructions?: string initialPastes?: PasteAttachment[] initialImages?: AttachedImage[] + initialFiles?: AttachedTextFile[] editingMessageIndex?: number | null onEditEnd?: () => void className?: string @@ -76,6 +87,7 @@ initialInstructions = '', initialPastes = undefined, initialImages = undefined, + initialFiles = undefined, editingMessageIndex = null, onEditEnd = () => {}, className = '', @@ -142,16 +154,22 @@ let contextTextareaComponent: ContextTextarea | undefined = $state() let instructionsTextareaComponent: HTMLTextAreaElement | undefined = $state() - let instructions = $state(untrack(() => initialInstructions)) + // The four lanes that ship with the next send — text, collapsed big-paste + // blobs, per-message images, per-message text files — owned by one draft so + // every aggregation applies the draft rules. The composer keeps only the + // async in-flight accounting (pending counters, byte reservations). + const draft = new MessageDraft( + untrack(() => ({ + text: initialInstructions, + pastes: initialPastes ?? [], + images: initialImages ?? [], + files: initialFiles ?? [] + })) + ) $effect(() => { - const text = instructions + const text = draft.text untrack(() => onDraftChange?.(text)) }) - // Collapsed big-paste blobs referenced by tokens in `instructions`. - let pastes = $state(untrack(() => initialPastes ?? [])) - // Per-message image attachments (drag/drop/paste), GLOBAL mode only. One-shot: - // they attach to the next send and clear, unlike the persistent attached-files store. - let images = $state(untrack(() => initialImages ?? [])) // Images being decoded right now. Holds off sending so a message can never go // out without an attachment the user already dropped, and reserves cap slots // against a concurrent drop. @@ -171,10 +189,10 @@ sendUserToast(`${model.model} can't read images. Switch to a vision model first.`, true) return } - // Count decodes already in flight: two drops that both read `images.length` + // Count decodes already in flight: two drops that both read the image count // before either resolves would each claim the same free slots and overshoot // the cap. - const remaining = MAX_ATTACHED_IMAGES - images.length - pendingImages + const remaining = MAX_ATTACHED_IMAGES - draft.images.length - pendingImages if (remaining <= 0) { sendUserToast(`You can attach up to ${MAX_ATTACHED_IMAGES} images.`, true) return @@ -210,7 +228,7 @@ failed++ } } - if (added.length > 0) images = [...images, ...added] + if (added.length > 0) draft.addImages(added) if (failed > 0) sendUserToast(`Could not attach ${failed} image(s).`, true) } finally { pendingImages -= batch.length @@ -218,7 +236,140 @@ } function removeImage(index: number) { - images = images.filter((_, i) => i !== index) + draft.images = draft.images.filter((_, i) => i !== index) + } + + // Files being read right now — same send-hold/slot-reservation role as pendingImages. + let pendingFiles = $state(0) + // Drop routing resolves file-system handles/entries asynchronously before it + // can call addTextFiles/addImages; a send during that window would land the + // dropped files on the NEXT message. Holds block sending (no slot or chip + // impact) until the drop handler finishes routing. + let ingestionHolds = $state(0) + export function holdSendForIngestion(): () => void { + ingestionHolds += 1 + let released = false + return () => { + if (!released) { + released = true + ingestionHolds -= 1 + } + } + } + // Bytes those in-flight reads have claimed against the conversation budget: + // two overlapping drops that both read the budget before either lands would + // otherwise each spend the same remaining allowance. + let pendingFileBytes = $state(0) + + // Publish this composer's staged bytes (committed attachments + in-flight + // reads) to the manager so a concurrently-mounted composer — the edit box + // while editing an earlier message — sees them in its own budget check and + // the two can't each spend the whole conversation allowance. + const composerKey = untrack(() => createLongHash()) + let stagedBytes = $derived( + draft.files.reduce((sum, f) => sum + textByteLength(f.content), 0) + pendingFileBytes + ) + $effect(() => { + aiChatManager.setComposerStaged(composerKey, editingMessageIndex, stagedBytes) + }) + $effect(() => () => aiChatManager.clearComposerStaged(composerKey)) + + /** Attach dropped/picked text files (sniffed + bounded). GLOBAL mode only. */ + export async function addTextFiles(candidates: File[]) { + if (aiChatManager.mode !== AIMode.GLOBAL) return + if (candidates.length === 0) return + const remaining = MAX_ATTACHED_FILES - draft.files.length - pendingFiles + if (remaining <= 0) { + sendUserToast(`You can attach up to ${MAX_ATTACHED_FILES} files.`, true) + return + } + const oversized = candidates.filter((f) => f.size > MAX_TEXT_FILE_BYTES) + if (oversized.length > 0) { + const mb = Math.round(MAX_TEXT_FILE_BYTES / 1_000_000) + sendUserToast( + `${oversized.length} file(s) over ${mb}MB were skipped — link their folder to read them on demand.`, + true + ) + } + const usable = candidates.filter((f) => f.size <= MAX_TEXT_FILE_BYTES) + if (usable.length === 0) return + let batch = usable.slice(0, remaining) + if (batch.length < usable.length) { + sendUserToast( + `You can attach up to ${MAX_ATTACHED_FILES} files; ${usable.length - batch.length} were skipped.`, + true + ) + } + // Conversation-level byte budget: transcript + queue + every live + // composer's stage (this one and, mid-edit, the other) + this composer's + // own pending reads. File content is persisted with every history save, so + // an unbounded total would grow the chat record without limit. The + // transcript sum skips any message a composer is editing — that composer's + // stage stands in for it, so counting both would charge those bytes twice. + let budget = + MAX_CONVERSATION_FILE_BYTES - + aiChatManager.attachmentBytesExcluding(composerKey) - + draft.files.reduce((sum, f) => sum + textByteLength(f.content), 0) - + pendingFileBytes + const withinBudget: File[] = [] + for (const f of batch) { + if (f.size <= budget) { + withinBudget.push(f) + budget -= f.size + } + } + if (withinBudget.length < batch.length) { + const mb = Math.round(MAX_CONVERSATION_FILE_BYTES / 1_000_000) + sendUserToast( + `${batch.length - withinBudget.length} file(s) skipped — this conversation reached its ${mb}MB attachment budget. Link a folder to read larger sets on demand.`, + true + ) + } + batch = withinBudget + if (batch.length === 0) return + pendingFiles += batch.length + const reservedBytes = batch.reduce((sum, f) => sum + f.size, 0) + pendingFileBytes += reservedBytes + try { + const reads: { name: string; content: string }[] = [] + let skipped = 0 + for (const file of batch) { + try { + const attached = await fileToAttachedTextFile(file) + if (attached) reads.push(attached) + else skipped++ + } catch { + skipped++ + } + } + // Commit through the draft in one synchronous step — fold (dedupe, + // courtesy rename) and decoded-byte admission both run against the live + // list, so another batch landing between this one's file reads can't be + // missed, and malformed input that inflates on decode can't slip past the + // raw-size admission above. This batch's own raw reservation is excluded + // from the budget — the decoded sizes replace it. + const liveBudget = + MAX_CONVERSATION_FILE_BYTES - + aiChatManager.attachmentBytesExcluding(composerKey) - + draft.files.reduce((sum, f) => sum + textByteLength(f.content), 0) - + (pendingFileBytes - reservedBytes) + const { droppedAtBudget } = draft.addFiles(reads, liveBudget) + if (droppedAtBudget > 0) { + const mb = Math.round(MAX_CONVERSATION_FILE_BYTES / 1_000_000) + sendUserToast( + `${droppedAtBudget} file(s) skipped — this conversation reached its ${mb}MB attachment budget. Link a folder to read larger sets on demand.`, + true + ) + } + if (skipped > 0) sendUserToast(`Skipped ${skipped} file(s) (non-text).`, true) + } finally { + pendingFiles -= batch.length + pendingFileBytes -= reservedBytes + } + } + + function removeFile(index: number) { + draft.files = draft.files.filter((_, i) => i !== index) } // App mode @ mention state @@ -250,9 +401,9 @@ * leave duplicate tokens. */ export function insertMention(title: string) { const target = `@${title}` - if (instructions.split(/\s+/).includes(target)) return - const sep = instructions.length === 0 || /\s$/.test(instructions) ? '' : ' ' - instructions = `${instructions}${sep}${target} ` + if (draft.text.split(/\s+/).includes(target)) return + const sep = draft.text.length === 0 || /\s$/.test(draft.text) ? '' : ' ' + draft.text = `${draft.text}${sep}${target} ` } /** Strip every `@title` token from the textarea — used when the user @@ -268,7 +419,7 @@ contextTextareaComponent?.unsyncMention(title) const escaped = title.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') const re = new RegExp(`(^|\\s)@${escaped}(\\s|$)`, 'g') - instructions = instructions.replace(re, (_m, lead, trail) => { + draft.text = draft.text.replace(re, (_m, lead, trail) => { // Boundary on at least one side → drop the mention entirely. if (!lead || !trail) return '' // Middle of text: keep ONE of the bracketing whitespace chars so @@ -296,12 +447,23 @@ export function restoreInstructions( value: string, restoredPastes: PasteAttachment[] = [], - restoredImages: AttachedImage[] = [] + restoredImages: AttachedImage[] = [], + restoredFiles: AttachedTextFile[] = [] ): boolean { - if (instructions.trim() || images.length > 0 || pendingImages > 0) return false - instructions = value - pastes = restoredPastes - images = restoredImages + // Attachments still decoding/reading (or mid-drop-routing) count as + // occupancy too — they belong to a draft the user started even though + // their lane is still empty. + if (pendingImages > 0 || pendingFiles > 0 || ingestionHolds > 0) return false + if ( + !draft.replaceIfEmpty({ + text: value, + pastes: restoredPastes, + images: restoredImages, + files: restoredFiles + }) + ) { + return false + } focusInput() return true } @@ -311,24 +473,30 @@ * the user typed is lost. Restored images join whatever is already * attached, up to the cap — dropping them would lose the attachment * silently, which is the whole reason the queue carries them. */ - export function prependText(text: string, restoredImages: AttachedImage[] = []): boolean { - // Whether the restored text landed on top of a draft the user was already - // writing: both instructions now share one composer, so the caller must keep - // both their contexts rather than replacing one with the other. - const mergedIntoDraft = !!text && !!instructions.trim() - // An image-only restore has empty text; prepending it would only add blank lines. - if (text) { - instructions = instructions.trim() ? `${text}\n\n${instructions}` : text + export function prependText( + text: string, + restoredImages: AttachedImage[] = [], + restoredFiles: AttachedTextFile[] = [] + ): boolean { + // mergedIntoDraft: the restored text landed on top of a draft the user was + // already writing — both instructions now share one composer, so the caller + // must keep both their contexts rather than replacing one with the other. + const { mergedIntoDraft, droppedImages, droppedFiles } = draft.prepend({ + text, + images: restoredImages, + files: restoredFiles + }) + if (droppedImages > 0) { + sendUserToast( + `You can attach up to ${MAX_ATTACHED_IMAGES} images; ${droppedImages} restored image(s) were dropped.`, + true + ) } - if (restoredImages.length > 0) { - const merged = [...images, ...restoredImages] - if (merged.length > MAX_ATTACHED_IMAGES) { - sendUserToast( - `You can attach up to ${MAX_ATTACHED_IMAGES} images; ${merged.length - MAX_ATTACHED_IMAGES} restored image(s) were dropped.`, - true - ) - } - images = merged.slice(0, MAX_ATTACHED_IMAGES) + if (droppedFiles > 0) { + sendUserToast( + `You can attach up to ${MAX_ATTACHED_FILES} files; ${droppedFiles} restored file(s) were dropped.`, + true + ) } focusInput() return mergedIntoDraft @@ -336,8 +504,8 @@ /** Insert a plain @filename mention for an attached file (used by the @ menu Files category). */ export function insertFileMention(name: string) { - const sep = instructions.length === 0 || instructions.endsWith(' ') ? '' : ' ' - instructions = `${instructions}${sep}${formatMention(name)} ` + const sep = draft.text.length === 0 || draft.text.endsWith(' ') ? '' : ' ' + draft.text = `${draft.text}${sep}${formatMention(name)} ` focusInput() } @@ -429,8 +597,8 @@ function sendRequest() { // The send button is disabled while decoding, but Enter reaches here directly. - // Sending now would drop the in-flight images onto the following message. - if (pendingImages > 0) { + // Sending now would drop the in-flight attachments onto the following message. + if (pendingImages > 0 || pendingFiles > 0 || ingestionHolds > 0) { return } if (aiChatManager.loading) { @@ -444,17 +612,16 @@ // chips picked at press time. if ( editingMessageIndex === null && - (instructions.trim() || - images.length > 0 || - (aiChatManager.mode === AIMode.GLOBAL && selectedContext.length > 0)) + (!draft.isEmpty || (aiChatManager.mode === AIMode.GLOBAL && selectedContext.length > 0)) ) { - aiChatManager.queueMessage(expanded(chatDraft(instructions, pastes)), images, [ - ...selectedContext - ]) + const sent = draft.take() + aiChatManager.queueMessage( + expanded(chatDraft(sent.text, sent.pastes)), + sent.images, + [...selectedContext], + sent.files + ) contextTextareaComponent?.clearForSend() - instructions = '' - pastes = [] - images = [] } return } @@ -462,25 +629,30 @@ // In edit mode selectedContext is the edit box's own copy (seeded from the // message's original chips), so send exactly what's shown — the user may // have added or removed chips. + const sent = draft.take() aiChatManager.restartGeneration( editingMessageIndex, - instructions, - pastes, - images, - selectedContext + sent.text, + sent.pastes, + sent.images, + selectedContext, + sent.files ) onEditEnd() } else { - aiChatManager.sendRequest({ instructions, pastes, images }) + const sent = draft.take() + aiChatManager.sendRequest({ + instructions: sent.text, + pastes: sent.pastes, + images: sent.images, + files: sent.files + }) // clearForSend() pre-zaps the textarea's mention-sync so the wipe // doesn't drop `selectedContext` before `AIChatManager.beforeSend` // snapshots it. Only mounted in SCRIPT/FLOW/GLOBAL — APP and the - // fallback textarea still rely on the plain `instructions = ''` - // reset (no `@`-mention state to coordinate). + // fallback textarea still rely on the draft reset alone (no + // `@`-mention state to coordinate). contextTextareaComponent?.clearForSend() - instructions = '' - pastes = [] - images = [] } } @@ -489,7 +661,7 @@ // for the conversation bubble and expands them for the LLM inside the manager. function submitRequest() { if (onSendRequest) { - onSendRequest(expanded(chatDraft(instructions, pastes))) + onSendRequest(expanded(chatDraft(draft.text, draft.pastes))) } else { sendRequest() } @@ -661,7 +833,7 @@ } function handleAppInput(_e: Event) { - const words = instructions.split(/\s+/) + const words = draft.text.split(/\s+/) const lastWord = words[words.length - 1] if ( @@ -680,9 +852,9 @@ function handleAppContextSelection(contextElement: ContextElement) { void addContextToSelection(contextElement) // Update instructions with the selected context title - const index = instructions.lastIndexOf('@') + const index = draft.text.lastIndexOf('@') if (index !== -1) { - instructions = instructions.substring(0, index) + `@${contextElement.title}` + draft.text = draft.text.substring(0, index) + `@${contextElement.title}` } showAppContextTooltip = false } @@ -696,7 +868,7 @@ {#snippet sendStopButton()} {@const isLoading = loading ?? aiChatManager.loading} - {@const emptyDraft = instructions.trim().length === 0 && images.length === 0} + {@const emptyDraft = draft.isEmpty} +{#snippet badgeRow()} + {@const contextChips = showContext ? selectedContext : domSelectorChips} + {#if contextChips.length > 0 || draft.files.length > 0 || pendingFiles > 0}
- {#each selectedContext as element (contextKey(element))} + {#each contextChips as element (contextKey(element))} { selectedContext = selectedContext?.filter((c) => !isSameContextElement(c, element)) - removeMention(element.title) + if (showContext) removeMention(element.title) }} /> {/each} -
- {/if} -{/snippet} - - -{#snippet domSelectorChipRow()} - {#if domSelectorChips.length > 0} -
- {#each domSelectorChips as element (contextKey(element))} + {#each draft.files as file, i (i)} { - selectedContext = selectedContext?.filter((c) => !isSameContextElement(c, element)) - }} + onDelete={() => removeFile(i)} /> {/each} + {#each { length: pendingFiles } as _, i (i)} +
+ +
+ {/each}
{/if} {/snippet} {#snippet imageChipsRow()} - {#if images.length > 0 || pendingImages > 0} -
- {#each images as image, i (i)} + {#if draft.images.length > 0 || pendingImages > 0} +
+ {#each draft.images as image, i (i)}
@@ -811,10 +987,13 @@
void addImages(files) + ? (pasted) => void addImages(pasted) + : undefined} + onTextFiles={aiChatManager.mode === AIMode.GLOBAL + ? (pasted) => void addTextFiles(pasted) : undefined} {availableContext} {selectedContext} @@ -833,16 +1012,7 @@ {onKeyDown} > {#snippet leading()} - {#if aiChatManager.mode === AIMode.GLOBAL} -
- -
- {/if} - {#if showContext} - {@render contextPickerRow()} - {:else} - {@render domSelectorChipRow()} - {/if} + {@render badgeRow()} {@render imageChipsRow()} {/snippet} @@ -854,12 +1024,12 @@
{:else if aiChatManager.mode === AIMode.APP} {#if showContext} - {@render contextPickerRow()} + {@render badgeRow()} {/if}