diff --git a/docs/plans/2026-03-14-v0.5-attachments-idle-design.md b/docs/plans/2026-03-14-v0.5-attachments-idle-design.md new file mode 100644 index 0000000..95a2375 --- /dev/null +++ b/docs/plans/2026-03-14-v0.5-attachments-idle-design.md @@ -0,0 +1,460 @@ +# Magnum Opus v0.5 — Attachments & IMAP IDLE + +**Goal:** Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE. No more polling delay for new mail. + +**Builds on:** v0.4 GTD email client (IMAP sync, SMTP compose, triage, tasks, ActionQueue, VTODO files). + +--- + +## Core Changes + +v0.5 adds three capabilities: + +1. **MIMEParser** — parse multipart MIME messages to extract attachments from received emails +2. **Attachment sending** — extend compose/SMTP to build multipart/mixed messages with file attachments +3. **IMAP IDLE** — persistent connection to INBOX for instant new-mail notifications + +--- + +## Architecture Overview + +``` +┌──────────────────────────────────────────────────────────────┐ +│ SwiftUI Apps │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ │ +│ │ ComposeView │ │ AttachStrip │ │ThreadDetailView│ │ +│ │ +file picker │ │ (download/ │ │ +inline images│ │ +│ └──────┬───────┘ │ preview) │ └───────┬───────┘ │ +│ │ └──────┬───────┘ │ │ +│ └─────────────────┼──────────────────┘ │ +│ ┌─────┴──────┐ │ +│ │ ViewModels │ │ +│ └─────┬──────┘ │ +├───────────────────────────┼──────────────────────────────────┤ +│ MagnumOpusCore (Swift Package) │ +│ ┌─────┴──────┐ │ +│ │ SyncEngine │ │ +│ │ │ │ +│ ┌───────────┼─────────┐ │ │ +│ │ ActionQueue │ │ │ +│ └──┬─────────┬───────┘ │ │ +│ ┌────────┘ └───────┐ │ │ +│ │ SMTPClient │ │IMAPClient│ │ ┌──────────┐ │ +│ │(multipart) │ │ │ │ │ MailStore │ │ +│ └────────────┘ │ │ │ │ (GRDB) │ │ +│ │ │ │ └──────────┘ │ +│ └──────────┘ │ │ +│ ┌──────────┐ │ ┌──────────┐ │ +│ │IMAPIdle │ │ │MIMEParser│ │ +│ │Client │ │ │ │ │ +│ │(NEW) │ │ │(NEW) │ │ +│ └──────────┘ │ └──────────┘ │ +└───────────────────┼─────────────┼────────────────────────────┘ + │ │ + ┌─────────┘ └──────────┐ + ▼ ▼ + ┌──────────┐ ┌──────────┐ + │ IMAP │ ← IDLE connection │ SMTP │ + │ Server │ + sync connection │ Server │ + └──────────┘ └──────────┘ +``` + +--- + +## MIMEParser Module + +New module in MagnumOpusCore. Parses RFC 2045/2046 MIME messages. No external dependencies — MIME is line-based. + +### Public API + +```swift +public enum MIMEParser { + /// Parse a raw MIME message into a structured tree of parts + public static func parse(_ rawMessage: String) -> MIMEMessage + + /// Decode content based on Content-Transfer-Encoding + public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data + + /// Generate a unique MIME boundary string + public static func generateBoundary() -> String +} + +public struct MIMEMessage: Sendable { + public var headers: [String: String] + public var parts: [MIMEPart] + public var textBody: String? // first text/plain part + public var htmlBody: String? // first text/html part + public var attachments: [MIMEAttachment] // non-inline parts + public var inlineImages: [MIMEAttachment] // Content-Disposition: inline +} + +public struct MIMEPart: Sendable { + public var headers: [String: String] + public var contentType: String // e.g., "text/plain" + public var charset: String? // e.g., "utf-8" + public var transferEncoding: TransferEncoding + public var disposition: ContentDisposition? + public var filename: String? + public var contentId: String? // for inline images (cid: references) + public var body: String // raw encoded content + public var subparts: [MIMEPart] // for nested multipart +} + +public struct MIMEAttachment: Sendable { + public var filename: String + public var mimeType: String + public var size: Int // decoded size in bytes + public var contentId: String? // for inline images + public var sectionPath: String // IMAP section number (e.g., "1.2") + public var isInline: Bool +} + +public enum TransferEncoding: String, Sendable { + case base64 + case quotedPrintable = "quoted-printable" + case sevenBit = "7bit" + case eightBit = "8bit" + case binary +} + +public enum ContentDisposition: Sendable { + case inline + case attachment +} +``` + +### Parsing Logic + +1. Parse top-level headers (split on first empty line) +2. Check `Content-Type` for `multipart/*` — if so, extract boundary and split body on boundary markers +3. Recursively parse each part +4. For `multipart/alternative`: pick text/plain and text/html parts for body +5. For `multipart/mixed`: first text part is body, rest are attachments +6. For nested structures (`multipart/mixed` containing `multipart/alternative`): recurse +7. Extract `filename` from `Content-Disposition: attachment; filename="..."` or `Content-Type: ...; name="..."` +8. Calculate decoded size from base64 content length (base64 inflates ~33%) +9. Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.) + +### Content Decoding + +- **base64:** standard Foundation `Data(base64Encoded:)` +- **quoted-printable:** reuse existing `quotedPrintableEncode` logic from MessageFormatter, add matching decode +- **7bit/8bit:** pass through as UTF-8 + +--- + +## Attachment Receiving + +### IMAP Sync Integration + +During body fetch in `SyncCoordinator`, replace the current raw body storage with MIME-aware parsing: + +1. Fetch full message via `IMAPClient.fetchBody(uid:)` (existing method — may need to fetch full RFC822 instead of just body text) +2. Parse with `MIMEParser.parse(rawMessage)` +3. Store `textBody` → `message.bodyText`, `htmlBody` → `message.bodyHtml` +4. For each attachment in `MIMEMessage.attachments` + `inlineImages`: + - Create `AttachmentRecord` with filename, mimeType, size, contentId, sectionPath + - Insert into `attachment` table (existing table, already has the right columns) +5. Set `message.hasAttachments = true` if any attachments found (field exists but not populated) + +### IMAPClient Extension + +New method for on-demand attachment download: + +```swift +func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data +``` + +Uses IMAP `BODY[section]` fetch to retrieve a specific MIME part by section number. Returns decoded content (base64 decoded). + +### Attachment Storage + +Downloaded attachments cached at: +`~/Library/Application Support/MagnumOpus//attachments/.` + +The `attachment.cachePath` column (existing, currently unused) is updated when content is downloaded. + +### Schema Changes + +No new tables needed. The existing `attachment` table has all required columns: +- id, messageId, filename, mimeType, size, contentId, cachePath + +Add one column: + +```sql +ALTER TABLE attachment ADD COLUMN sectionPath TEXT; +``` + +This stores the IMAP section number for on-demand fetch. + +### Attachment Download Flow + +1. User taps attachment chip → check `cachePath` +2. If cached: open with QuickLook +3. If not cached: + - Show download progress + - Call `IMAPClient.fetchSection(uid:mailbox:section:)` with the stored `sectionPath` + - Save to cache directory + - Update `attachment.cachePath` + - Open with QuickLook + +--- + +## Attachment Sending + +### Compose Flow Changes + +**ComposeViewModel additions:** +- `attachments: [(url: URL, filename: String, mimeType: String, data: Data)]` +- `attachFile()` — triggered by file picker, reads file data, adds to list +- `removeAttachment(at:)` — remove from list + +**ComposeView additions:** +- "Attach" toolbar button + `⌘⇧A` shortcut → `.fileImporter` system file picker +- Attachment chips below the body editor: filename + size + remove button +- Allow multiple file selection + +### MessageFormatter Extension + +When attachments are present, build a `multipart/mixed` message instead of single-part: + +``` +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary="----MagnumOpus-" + +------MagnumOpus- +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + + +------MagnumOpus- +Content-Type: application/pdf; name="report.pdf" +Content-Disposition: attachment; filename="report.pdf" +Content-Transfer-Encoding: base64 + + +------MagnumOpus--- +``` + +When no attachments, keep the current simple single-part format unchanged. + +New public methods: + +```swift +public static func formatMultipart( + _ message: OutgoingMessage, + attachments: [(filename: String, mimeType: String, data: Data)] +) -> String + +public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String +``` + +### MIME Type Detection + +Use `UTType` (UniformTypeIdentifiers framework) to detect MIME type from file extension. Fallback: `application/octet-stream`. + +--- + +## IMAP IDLE + +### IMAPIdleClient + +New actor in the IMAPClient module. Manages a dedicated, long-lived IMAP connection for IDLE on INBOX. + +```swift +public actor IMAPIdleClient: Sendable { + public init(host: String, port: Int, credentials: Credentials) + + /// Start monitoring INBOX. Calls onNewMail when server sends EXISTS. + public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws + + /// Stop monitoring and disconnect. + public func stopMonitoring() async +} +``` + +### IDLE Protocol Sequence + +``` +→ LOGIN (authenticate) +→ SELECT INBOX +→ IDLE +← + idling + ... server holds connection ... +← * 42 EXISTS ← new mail arrived +→ DONE ← break IDLE +← OK IDLE terminated +→ [trigger sync] +→ IDLE ← re-enter IDLE + ... repeat ... +``` + +### Lifecycle Management + +- **Start:** After successful first sync, if server advertises `IDLE` capability +- **Re-IDLE timer:** Send `DONE` + re-issue `IDLE` every 29 minutes (RFC 2177) +- **Reconnect:** On connection drop, reconnect with exponential backoff (5s, 10s, 30s, 1m, 5m cap) +- **Stop:** On app background (iOS), window close (macOS), or explicit stop +- **Fallback:** If server doesn't support IDLE, keep existing periodic poll (bumped to 15 min) + +### SyncCoordinator Integration + +```swift +// New properties +private var idleClient: IMAPIdleClient? +private var idleTask: Task? + +// New methods +public func startIdleMonitoring() // check capabilities, start if supported +public func stopIdleMonitoring() // stop IDLE, cancel task + +// In startIdleMonitoring: +// 1. Check server capabilities for "IDLE" +// 2. Create IMAPIdleClient with same host/port/credentials +// 3. Call startMonitoring(onNewMail: { self.syncNow() }) +// 4. Adjust periodic sync interval to 15 minutes (fallback) +``` + +### NIO Implementation + +`IMAPIdleClient` uses the same NIO patterns as `IMAPClient`: +- `IMAPConnection` for the TLS channel (can reuse existing class) +- `IMAPCommandRunner` for LOGIN/SELECT (can reuse) +- Custom IDLE handler that watches for untagged `* EXISTS` responses +- Use `CheckedContinuation` for async notification delivery + +The IDLE connection is separate from the sync connection — two concurrent connections to the IMAP server. + +--- + +## Attachment UI + +### Thread Detail — Attachment Strip + +Below each message body that has attachments, show a horizontal strip of attachment chips: + +``` +┌──────────────────────────────────────────────┐ +│ 📎 report.pdf (2.1 MB) 📎 photo.jpg (450 KB) │ +│ [tap to download] [tap to preview] │ +└──────────────────────────────────────────────┘ +``` + +Each chip shows: +- File type icon (SF Symbol based on MIME type: doc.text, photo, film, music.note, doc) +- Filename +- Size +- Download state: not downloaded → downloading (progress) → cached (tap to preview) + +**Inline images:** For HTML messages, replace `cid:` URLs with the downloaded image data. If the inline image isn't cached, show a placeholder with a download button. + +### Compose — Attachment Chips + +Below the body TextEditor, attached files show as removable chips: + +``` +┌──────────────────────────────────────────────┐ +│ 📎 report.pdf (2.1 MB) ✕ 📎 data.csv (15 KB) ✕ │ +└──────────────────────────────────────────────┘ +``` + +Tap ✕ to remove. The "Attach" button in the toolbar opens `.fileImporter`. + +### Message List — Paperclip Icon + +Thread rows show a 📎 icon when any message in the thread has attachments. Uses the existing `hasAttachments` field on `MessageSummary`. + +--- + +## Testing Strategy + +### MIMEParser Tests + +- Parse single-part text/plain message → correct body, no attachments +- Parse multipart/mixed with text + PDF → text body extracted, one attachment with correct metadata +- Parse multipart/alternative (text + html) → both bodies extracted, no attachments +- Parse nested multipart/mixed containing multipart/alternative → correct body + attachments +- Base64 decode → correct binary output +- Quoted-printable decode → correct text with non-ASCII +- Extract filename from Content-Disposition and Content-Type name parameter +- Section path assignment (1, 1.1, 1.2, 2) +- Handle missing Content-Disposition (infer attachment from non-text content type) +- Handle malformed MIME (missing boundary, truncated parts) gracefully + +### MessageFormatter Tests (extended) + +- Format with attachments → valid multipart/mixed output +- Format without attachments → single-part text/plain (no multipart wrapper) +- Base64 encoding wraps at 76 characters +- Boundary string is unique and doesn't appear in content +- Multiple attachments produce correct number of parts +- Attachment content-type and filename headers correct + +### IMAP IDLE Tests + +- MockIMAPClient extended with IDLE simulation +- Start monitoring triggers IDLE command +- EXISTS response triggers onNewMail callback +- 29-minute re-IDLE timer fires correctly +- Connection drop triggers reconnect +- Server without IDLE capability → monitoring not started, returns gracefully +- Stop monitoring sends DONE and disconnects + +### Integration Tests + +- Sync with MIME message populates attachment table +- Attachment download fetches correct section and caches +- Compose with attachment produces valid multipart message via SMTP +- IDLE notification triggers sync cycle + +--- + +## Dependencies + +No new external dependencies. Uses existing packages + Foundation's `UTType` for MIME type detection. + +| Package | Purpose | +|---------|---------| +| swift-nio-imap | IMAP protocol including IDLE (existing) | +| swift-nio | Networking (existing) | +| swift-nio-ssl | TLS (existing) | +| GRDB.swift | SQLite (existing) | +| UniformTypeIdentifiers | MIME type detection (Apple framework, no package needed) | + +--- + +## Schema Changes + +### Migration v4_attachmentSection + +```sql +ALTER TABLE attachment ADD COLUMN sectionPath TEXT; +``` + +--- + +## v0.5 Scope + +### In + +- MIMEParser module (multipart parsing, content decoding, section paths) +- Attachment receiving: MIME-aware sync, attachment table population, on-demand download, cache +- Attachment sending: file picker in compose, multipart/mixed message formatting, base64 encoding +- Attachment UI: attachment strip in thread detail, inline image rendering, compose chips, paperclip icon +- IMAP IDLE: dedicated connection, EXISTS notification, auto-sync trigger, 29-min re-IDLE, reconnect +- Periodic sync interval bumped to 15 min (IDLE is the fast path) +- Schema migration for attachment sectionPath + +### Out (Deferred) + +- Delegation / waiting loop (v0.5+) +- Tags / GTD contexts (v0.5+) +- Defer to time-of-day / notifications (v0.5+) +- Task search in FTS5 (v0.5+) +- CalDAV sync (v0.6) +- Task dependencies (v0.6) +- Rich text compose (future) +- Multiple accounts (future) +- Drag-and-drop attachments (future) +- Attachment size limits / compression (future)