diff --git a/docs/plans/2026-03-14-v0.5-attachments-idle-design.md b/docs/plans/2026-03-14-v0.5-attachments-idle-design.md index 95a2375..42550cb 100644 --- a/docs/plans/2026-03-14-v0.5-attachments-idle-design.md +++ b/docs/plans/2026-03-14-v0.5-attachments-idle-design.md @@ -97,14 +97,14 @@ public struct MIMEPart: Sendable { public var disposition: ContentDisposition? public var filename: String? public var contentId: String? // for inline images (cid: references) - public var body: String // raw encoded content + public var body: Data // raw content (decoded from transfer encoding) public var subparts: [MIMEPart] // for nested multipart } public struct MIMEAttachment: Sendable { public var filename: String public var mimeType: String - public var size: Int // decoded size in bytes + public var size: Int // estimated decoded size (base64 length * 3/4) public var contentId: String? // for inline images public var sectionPath: String // IMAP section number (e.g., "1.2") public var isInline: Bool @@ -130,11 +130,20 @@ public enum ContentDisposition: Sendable { 2. Check `Content-Type` for `multipart/*` — if so, extract boundary and split body on boundary markers 3. Recursively parse each part 4. For `multipart/alternative`: pick text/plain and text/html parts for body -5. For `multipart/mixed`: first text part is body, rest are attachments -6. For nested structures (`multipart/mixed` containing `multipart/alternative`): recurse -7. Extract `filename` from `Content-Disposition: attachment; filename="..."` or `Content-Type: ...; name="..."` -8. Calculate decoded size from base64 content length (base64 inflates ~33%) -9. Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.) +5. For `multipart/mixed`: first text part (or multipart/alternative) is body, rest are attachments +6. For `multipart/related`: HTML body + inline images (Content-ID references for `cid:` URLs). This is the standard container for HTML newsletters with embedded images. +7. For nested structures (e.g., `multipart/mixed` containing `multipart/related` containing `multipart/alternative`): recurse +8. Extract `filename` from `Content-Disposition: attachment; filename="..."` or `Content-Type: ...; name="..."`. Decode RFC 2047 encoded words (`=?charset?encoding?text?=`) for non-ASCII filenames. +9. Estimate decoded size from base64 content length (base64 inflates ~33%): `encodedLength * 3 / 4` +10. Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.) + +### Boundary Generation + +`MIMEParser.generateBoundary()` returns `"=_MagnumOpus_\(UUID().uuidString)"`. The `=_` prefix makes collision with base64 content essentially impossible (base64 alphabet doesn't include `=_` at line starts). + +### RFC 2047 Decoding + +Filenames may be encoded as `=?utf-8?B??=` or `=?utf-8?Q??=`. The parser detects this pattern and decodes to produce human-readable filenames. ### Content Decoding @@ -148,15 +157,25 @@ public enum ContentDisposition: Sendable { ### IMAP Sync Integration +The existing `IMAPClient.fetchBody(uid:)` fetches only body text (BODY[TEXT]), which strips headers needed for MIME parsing. Replace with a new method that fetches the full RFC822 message: + +```swift +func fetchFullMessage(uid: Int) async throws -> String +``` + +This uses `BODY.PEEK[]` (full message including headers) instead of `BODY[TEXT]`. The existing `fetchBody(uid:)` remains available for non-MIME use cases. + During body fetch in `SyncCoordinator`, replace the current raw body storage with MIME-aware parsing: -1. Fetch full message via `IMAPClient.fetchBody(uid:)` (existing method — may need to fetch full RFC822 instead of just body text) +1. Fetch full message via `IMAPClient.fetchFullMessage(uid:)` 2. Parse with `MIMEParser.parse(rawMessage)` 3. Store `textBody` → `message.bodyText`, `htmlBody` → `message.bodyHtml` 4. For each attachment in `MIMEMessage.attachments` + `inlineImages`: - Create `AttachmentRecord` with filename, mimeType, size, contentId, sectionPath - Insert into `attachment` table (existing table, already has the right columns) -5. Set `message.hasAttachments = true` if any attachments found (field exists but not populated) +5. Update `message.hasAttachments` (see migration below) + +**Note on fetchEnvelopes:** The existing `fetchEnvelopes` method also fetches body text inline during initial sync. This path should also be updated to use MIME-aware parsing, or body fetching should be deferred to the separate `prefetchBodies` phase (which already exists). The simpler approach: keep `fetchEnvelopes` fetching only envelopes (no body), and let `prefetchBodies` do the full RFC822 fetch with MIME parsing. This avoids doubling the bandwidth during initial sync. ### IMAPClient Extension @@ -194,6 +213,7 @@ This stores the IMAP section number for on-demand fetch. 2. If cached: open with QuickLook 3. If not cached: - Show download progress + - Check UID validity: compare stored `mailbox.uidValidity` with server. If changed, UIDs are stale — show error "Message may have moved, please re-sync" - Call `IMAPClient.fetchSection(uid:mailbox:section:)` with the stored `sectionPath` - Save to cache directory - Update `attachment.cachePath` @@ -207,8 +227,9 @@ This stores the IMAP section number for on-demand fetch. **ComposeViewModel additions:** - `attachments: [(url: URL, filename: String, mimeType: String, data: Data)]` -- `attachFile()` — triggered by file picker, reads file data, adds to list +- `attachFile()` — triggered by file picker, reads file data, adds to list. Refuses files > 25 MB with user-facing error (the SMTP server will ultimately enforce limits, but loading a 500 MB video into memory crashes the app). - `removeAttachment(at:)` — remove from list +- `totalAttachmentSize: Int` — computed sum, shown in UI **ComposeView additions:** - "Attach" toolbar button + `⌘⇧A` shortcut → `.fileImporter` system file picker @@ -239,6 +260,8 @@ Content-Transfer-Encoding: base64 When no attachments, keep the current simple single-part format unchanged. +The existing `MessageFormatter.format(_:)` is updated to accept an optional attachments parameter. If attachments are present, it delegates to `formatMultipart`; otherwise it produces the existing single-part output. The ActionQueue/SMTP send path calls `format()` as before — no changes needed downstream. + New public methods: ```swift @@ -319,13 +342,40 @@ public func stopIdleMonitoring() // stop IDLE, cancel task ### NIO Implementation `IMAPIdleClient` uses the same NIO patterns as `IMAPClient`: -- `IMAPConnection` for the TLS channel (can reuse existing class) -- `IMAPCommandRunner` for LOGIN/SELECT (can reuse) -- Custom IDLE handler that watches for untagged `* EXISTS` responses -- Use `CheckedContinuation` for async notification delivery +- `IMAPConnection` for the TLS channel (reuse existing class) +- `IMAPCommandRunner` for LOGIN/SELECT (reuse) +- **New: `IMAPIdleHandler`** — a separate `ChannelInboundHandler` that replaces the standard `IMAPResponseHandler` after entering IDLE mode. Unlike the standard handler (which holds a single `CheckedContinuation` resolved on tagged responses), `IMAPIdleHandler` uses an `AsyncStream` to deliver a stream of untagged responses. The IDLE loop consumes this stream: + +```swift +enum IMAPIdleEvent: Sendable { + case exists(Int) // * EXISTS — new mail + case expunge(Int) // * EXPUNGE — message removed + case idleTerminated // OK IDLE terminated (after DONE) +} +``` + +The IDLE loop in `IMAPIdleClient`: +1. Send `IDLE` command +2. Consume `AsyncStream` in a for-await loop +3. On `.exists`: call `onNewMail()`, send `DONE`, wait for `.idleTerminated`, re-enter IDLE +4. On connection drop: the stream ends, trigger reconnect + +This avoids the single-continuation limitation of the standard response handler. The IDLE connection is separate from the sync connection — two concurrent connections to the IMAP server. +### INBOX Resolution + +The mailbox name for IDLE is resolved via `MailStore.mailboxWithRole("inbox", accountId:)` (using the role detection from v0.3). If no inbox role is found, falls back to the literal string "INBOX" (RFC 3501 requires this name). + +### Capability Check + +IDLE capability is checked via the sync `IMAPClient` (which caches capabilities from the initial connection). `startIdleMonitoring()` reads this cached value — it does not require the IDLE connection to query capabilities separately. + +### iOS Background Behavior + +IMAP IDLE is **foreground-only on iOS**. When the app enters background, `stopIdleMonitoring()` is called. On foreground resume, a fresh `syncNow()` is triggered to catch any mail that arrived while backgrounded, then IDLE monitoring restarts. On macOS, IDLE runs continuously while the app window is open. + --- ## Attachment UI @@ -381,6 +431,8 @@ Thread rows show a 📎 icon when any message in the thread has attachments. Use - Section path assignment (1, 1.1, 1.2, 2) - Handle missing Content-Disposition (infer attachment from non-text content type) - Handle malformed MIME (missing boundary, truncated parts) gracefully +- Parse multipart/related (HTML + inline images with cid: references) +- Decode RFC 2047 encoded filenames (=?utf-8?B?...?= and =?utf-8?Q?...?=) ### MessageFormatter Tests (extended) @@ -426,12 +478,17 @@ No new external dependencies. Uses existing packages + Foundation's `UTType` for ## Schema Changes -### Migration v4_attachmentSection +### Migration v4_attachment: Attachment MIME support ```sql ALTER TABLE attachment ADD COLUMN sectionPath TEXT; +ALTER TABLE message ADD COLUMN hasAttachments INTEGER NOT NULL DEFAULT 0; ``` +The `hasAttachments` column is populated during MIME parsing. The `sectionPath` stores the IMAP section number for on-demand part fetch. + +Note: v4 follows after v3 migrations from v0.4 (v3_task, v3_label, v3_deferral). + --- ## v0.5 Scope @@ -444,7 +501,9 @@ ALTER TABLE attachment ADD COLUMN sectionPath TEXT; - Attachment UI: attachment strip in thread detail, inline image rendering, compose chips, paperclip icon - IMAP IDLE: dedicated connection, EXISTS notification, auto-sync trigger, 29-min re-IDLE, reconnect - Periodic sync interval bumped to 15 min (IDLE is the fast path) -- Schema migration for attachment sectionPath +- Schema migration for attachment sectionPath + message hasAttachments +- RFC 2047 filename decoding +- Per-file attachment size guard (25 MB) ### Out (Deferred) @@ -457,4 +516,4 @@ ALTER TABLE attachment ADD COLUMN sectionPath TEXT; - Rich text compose (future) - Multiple accounts (future) - Drag-and-drop attachments (future) -- Attachment size limits / compression (future) +- Attachment compression (future)