add v0.5 design spec: attachments (send+receive), IMAP IDLE
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
460
docs/plans/2026-03-14-v0.5-attachments-idle-design.md
Normal file
460
docs/plans/2026-03-14-v0.5-attachments-idle-design.md
Normal file
@@ -0,0 +1,460 @@
|
||||
# Magnum Opus v0.5 — Attachments & IMAP IDLE
|
||||
|
||||
**Goal:** Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE. No more polling delay for new mail.
|
||||
|
||||
**Builds on:** v0.4 GTD email client (IMAP sync, SMTP compose, triage, tasks, ActionQueue, VTODO files).
|
||||
|
||||
---
|
||||
|
||||
## Core Changes
|
||||
|
||||
v0.5 adds three capabilities:
|
||||
|
||||
1. **MIMEParser** — parse multipart MIME messages to extract attachments from received emails
|
||||
2. **Attachment sending** — extend compose/SMTP to build multipart/mixed messages with file attachments
|
||||
3. **IMAP IDLE** — persistent connection to INBOX for instant new-mail notifications
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ SwiftUI Apps │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ │
|
||||
│ │ ComposeView │ │ AttachStrip │ │ThreadDetailView│ │
|
||||
│ │ +file picker │ │ (download/ │ │ +inline images│ │
|
||||
│ └──────┬───────┘ │ preview) │ └───────┬───────┘ │
|
||||
│ │ └──────┬───────┘ │ │
|
||||
│ └─────────────────┼──────────────────┘ │
|
||||
│ ┌─────┴──────┐ │
|
||||
│ │ ViewModels │ │
|
||||
│ └─────┬──────┘ │
|
||||
├───────────────────────────┼──────────────────────────────────┤
|
||||
│ MagnumOpusCore (Swift Package) │
|
||||
│ ┌─────┴──────┐ │
|
||||
│ │ SyncEngine │ │
|
||||
│ │ │ │
|
||||
│ ┌───────────┼─────────┐ │ │
|
||||
│ │ ActionQueue │ │ │
|
||||
│ └──┬─────────┬───────┘ │ │
|
||||
│ ┌────────┘ └───────┐ │ │
|
||||
│ │ SMTPClient │ │IMAPClient│ │ ┌──────────┐ │
|
||||
│ │(multipart) │ │ │ │ │ MailStore │ │
|
||||
│ └────────────┘ │ │ │ │ (GRDB) │ │
|
||||
│ │ │ │ └──────────┘ │
|
||||
│ └──────────┘ │ │
|
||||
│ ┌──────────┐ │ ┌──────────┐ │
|
||||
│ │IMAPIdle │ │ │MIMEParser│ │
|
||||
│ │Client │ │ │ │ │
|
||||
│ │(NEW) │ │ │(NEW) │ │
|
||||
│ └──────────┘ │ └──────────┘ │
|
||||
└───────────────────┼─────────────┼────────────────────────────┘
|
||||
│ │
|
||||
┌─────────┘ └──────────┐
|
||||
▼ ▼
|
||||
┌──────────┐ ┌──────────┐
|
||||
│ IMAP │ ← IDLE connection │ SMTP │
|
||||
│ Server │ + sync connection │ Server │
|
||||
└──────────┘ └──────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MIMEParser Module
|
||||
|
||||
New module in MagnumOpusCore. Parses RFC 2045/2046 MIME messages. No external dependencies — MIME is line-based.
|
||||
|
||||
### Public API
|
||||
|
||||
```swift
|
||||
public enum MIMEParser {
|
||||
/// Parse a raw MIME message into a structured tree of parts
|
||||
public static func parse(_ rawMessage: String) -> MIMEMessage
|
||||
|
||||
/// Decode content based on Content-Transfer-Encoding
|
||||
public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data
|
||||
|
||||
/// Generate a unique MIME boundary string
|
||||
public static func generateBoundary() -> String
|
||||
}
|
||||
|
||||
public struct MIMEMessage: Sendable {
|
||||
public var headers: [String: String]
|
||||
public var parts: [MIMEPart]
|
||||
public var textBody: String? // first text/plain part
|
||||
public var htmlBody: String? // first text/html part
|
||||
public var attachments: [MIMEAttachment] // non-inline parts
|
||||
public var inlineImages: [MIMEAttachment] // Content-Disposition: inline
|
||||
}
|
||||
|
||||
public struct MIMEPart: Sendable {
|
||||
public var headers: [String: String]
|
||||
public var contentType: String // e.g., "text/plain"
|
||||
public var charset: String? // e.g., "utf-8"
|
||||
public var transferEncoding: TransferEncoding
|
||||
public var disposition: ContentDisposition?
|
||||
public var filename: String?
|
||||
public var contentId: String? // for inline images (cid: references)
|
||||
public var body: String // raw encoded content
|
||||
public var subparts: [MIMEPart] // for nested multipart
|
||||
}
|
||||
|
||||
public struct MIMEAttachment: Sendable {
|
||||
public var filename: String
|
||||
public var mimeType: String
|
||||
public var size: Int // decoded size in bytes
|
||||
public var contentId: String? // for inline images
|
||||
public var sectionPath: String // IMAP section number (e.g., "1.2")
|
||||
public var isInline: Bool
|
||||
}
|
||||
|
||||
public enum TransferEncoding: String, Sendable {
|
||||
case base64
|
||||
case quotedPrintable = "quoted-printable"
|
||||
case sevenBit = "7bit"
|
||||
case eightBit = "8bit"
|
||||
case binary
|
||||
}
|
||||
|
||||
public enum ContentDisposition: Sendable {
|
||||
case inline
|
||||
case attachment
|
||||
}
|
||||
```
|
||||
|
||||
### Parsing Logic
|
||||
|
||||
1. Parse top-level headers (split on first empty line)
|
||||
2. Check `Content-Type` for `multipart/*` — if so, extract boundary and split body on boundary markers
|
||||
3. Recursively parse each part
|
||||
4. For `multipart/alternative`: pick text/plain and text/html parts for body
|
||||
5. For `multipart/mixed`: first text part is body, rest are attachments
|
||||
6. For nested structures (`multipart/mixed` containing `multipart/alternative`): recurse
|
||||
7. Extract `filename` from `Content-Disposition: attachment; filename="..."` or `Content-Type: ...; name="..."`
|
||||
8. Calculate decoded size from base64 content length (base64 inflates ~33%)
|
||||
9. Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.)
|
||||
|
||||
### Content Decoding
|
||||
|
||||
- **base64:** standard Foundation `Data(base64Encoded:)`
|
||||
- **quoted-printable:** reuse existing `quotedPrintableEncode` logic from MessageFormatter, add matching decode
|
||||
- **7bit/8bit:** pass through as UTF-8
|
||||
|
||||
---
|
||||
|
||||
## Attachment Receiving
|
||||
|
||||
### IMAP Sync Integration
|
||||
|
||||
During body fetch in `SyncCoordinator`, replace the current raw body storage with MIME-aware parsing:
|
||||
|
||||
1. Fetch full message via `IMAPClient.fetchBody(uid:)` (existing method — may need to fetch full RFC822 instead of just body text)
|
||||
2. Parse with `MIMEParser.parse(rawMessage)`
|
||||
3. Store `textBody` → `message.bodyText`, `htmlBody` → `message.bodyHtml`
|
||||
4. For each attachment in `MIMEMessage.attachments` + `inlineImages`:
|
||||
- Create `AttachmentRecord` with filename, mimeType, size, contentId, sectionPath
|
||||
- Insert into `attachment` table (existing table, already has the right columns)
|
||||
5. Set `message.hasAttachments = true` if any attachments found (field exists but not populated)
|
||||
|
||||
### IMAPClient Extension
|
||||
|
||||
New method for on-demand attachment download:
|
||||
|
||||
```swift
|
||||
func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data
|
||||
```
|
||||
|
||||
Uses IMAP `BODY[section]` fetch to retrieve a specific MIME part by section number. Returns decoded content (base64 decoded).
|
||||
|
||||
### Attachment Storage
|
||||
|
||||
Downloaded attachments cached at:
|
||||
`~/Library/Application Support/MagnumOpus/<accountId>/attachments/<attachmentId>.<ext>`
|
||||
|
||||
The `attachment.cachePath` column (existing, currently unused) is updated when content is downloaded.
|
||||
|
||||
### Schema Changes
|
||||
|
||||
No new tables needed. The existing `attachment` table has all required columns:
|
||||
- id, messageId, filename, mimeType, size, contentId, cachePath
|
||||
|
||||
Add one column:
|
||||
|
||||
```sql
|
||||
ALTER TABLE attachment ADD COLUMN sectionPath TEXT;
|
||||
```
|
||||
|
||||
This stores the IMAP section number for on-demand fetch.
|
||||
|
||||
### Attachment Download Flow
|
||||
|
||||
1. User taps attachment chip → check `cachePath`
|
||||
2. If cached: open with QuickLook
|
||||
3. If not cached:
|
||||
- Show download progress
|
||||
- Call `IMAPClient.fetchSection(uid:mailbox:section:)` with the stored `sectionPath`
|
||||
- Save to cache directory
|
||||
- Update `attachment.cachePath`
|
||||
- Open with QuickLook
|
||||
|
||||
---
|
||||
|
||||
## Attachment Sending
|
||||
|
||||
### Compose Flow Changes
|
||||
|
||||
**ComposeViewModel additions:**
|
||||
- `attachments: [(url: URL, filename: String, mimeType: String, data: Data)]`
|
||||
- `attachFile()` — triggered by file picker, reads file data, adds to list
|
||||
- `removeAttachment(at:)` — remove from list
|
||||
|
||||
**ComposeView additions:**
|
||||
- "Attach" toolbar button + `⌘⇧A` shortcut → `.fileImporter` system file picker
|
||||
- Attachment chips below the body editor: filename + size + remove button
|
||||
- Allow multiple file selection
|
||||
|
||||
### MessageFormatter Extension
|
||||
|
||||
When attachments are present, build a `multipart/mixed` message instead of single-part:
|
||||
|
||||
```
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/mixed; boundary="----MagnumOpus-<uuid>"
|
||||
|
||||
------MagnumOpus-<uuid>
|
||||
Content-Type: text/plain; charset=utf-8
|
||||
Content-Transfer-Encoding: quoted-printable
|
||||
|
||||
<body text, quoted-printable encoded>
|
||||
------MagnumOpus-<uuid>
|
||||
Content-Type: application/pdf; name="report.pdf"
|
||||
Content-Disposition: attachment; filename="report.pdf"
|
||||
Content-Transfer-Encoding: base64
|
||||
|
||||
<base64 encoded file content, wrapped at 76 chars>
|
||||
------MagnumOpus-<uuid>--
|
||||
```
|
||||
|
||||
When no attachments, keep the current simple single-part format unchanged.
|
||||
|
||||
New public methods:
|
||||
|
||||
```swift
|
||||
public static func formatMultipart(
|
||||
_ message: OutgoingMessage,
|
||||
attachments: [(filename: String, mimeType: String, data: Data)]
|
||||
) -> String
|
||||
|
||||
public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String
|
||||
```
|
||||
|
||||
### MIME Type Detection
|
||||
|
||||
Use `UTType` (UniformTypeIdentifiers framework) to detect MIME type from file extension. Fallback: `application/octet-stream`.
|
||||
|
||||
---
|
||||
|
||||
## IMAP IDLE
|
||||
|
||||
### IMAPIdleClient
|
||||
|
||||
New actor in the IMAPClient module. Manages a dedicated, long-lived IMAP connection for IDLE on INBOX.
|
||||
|
||||
```swift
|
||||
public actor IMAPIdleClient: Sendable {
|
||||
public init(host: String, port: Int, credentials: Credentials)
|
||||
|
||||
/// Start monitoring INBOX. Calls onNewMail when server sends EXISTS.
|
||||
public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws
|
||||
|
||||
/// Stop monitoring and disconnect.
|
||||
public func stopMonitoring() async
|
||||
}
|
||||
```
|
||||
|
||||
### IDLE Protocol Sequence
|
||||
|
||||
```
|
||||
→ LOGIN (authenticate)
|
||||
→ SELECT INBOX
|
||||
→ IDLE
|
||||
← + idling
|
||||
... server holds connection ...
|
||||
← * 42 EXISTS ← new mail arrived
|
||||
→ DONE ← break IDLE
|
||||
← OK IDLE terminated
|
||||
→ [trigger sync]
|
||||
→ IDLE ← re-enter IDLE
|
||||
... repeat ...
|
||||
```
|
||||
|
||||
### Lifecycle Management
|
||||
|
||||
- **Start:** After successful first sync, if server advertises `IDLE` capability
|
||||
- **Re-IDLE timer:** Send `DONE` + re-issue `IDLE` every 29 minutes (RFC 2177)
|
||||
- **Reconnect:** On connection drop, reconnect with exponential backoff (5s, 10s, 30s, 1m, 5m cap)
|
||||
- **Stop:** On app background (iOS), window close (macOS), or explicit stop
|
||||
- **Fallback:** If server doesn't support IDLE, keep existing periodic poll (bumped to 15 min)
|
||||
|
||||
### SyncCoordinator Integration
|
||||
|
||||
```swift
|
||||
// New properties
|
||||
private var idleClient: IMAPIdleClient?
|
||||
private var idleTask: Task<Void, Never>?
|
||||
|
||||
// New methods
|
||||
public func startIdleMonitoring() // check capabilities, start if supported
|
||||
public func stopIdleMonitoring() // stop IDLE, cancel task
|
||||
|
||||
// In startIdleMonitoring:
|
||||
// 1. Check server capabilities for "IDLE"
|
||||
// 2. Create IMAPIdleClient with same host/port/credentials
|
||||
// 3. Call startMonitoring(onNewMail: { self.syncNow() })
|
||||
// 4. Adjust periodic sync interval to 15 minutes (fallback)
|
||||
```
|
||||
|
||||
### NIO Implementation
|
||||
|
||||
`IMAPIdleClient` uses the same NIO patterns as `IMAPClient`:
|
||||
- `IMAPConnection` for the TLS channel (can reuse existing class)
|
||||
- `IMAPCommandRunner` for LOGIN/SELECT (can reuse)
|
||||
- Custom IDLE handler that watches for untagged `* <n> EXISTS` responses
|
||||
- Use `CheckedContinuation` for async notification delivery
|
||||
|
||||
The IDLE connection is separate from the sync connection — two concurrent connections to the IMAP server.
|
||||
|
||||
---
|
||||
|
||||
## Attachment UI
|
||||
|
||||
### Thread Detail — Attachment Strip
|
||||
|
||||
Below each message body that has attachments, show a horizontal strip of attachment chips:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ 📎 report.pdf (2.1 MB) 📎 photo.jpg (450 KB) │
|
||||
│ [tap to download] [tap to preview] │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Each chip shows:
|
||||
- File type icon (SF Symbol based on MIME type: doc.text, photo, film, music.note, doc)
|
||||
- Filename
|
||||
- Size
|
||||
- Download state: not downloaded → downloading (progress) → cached (tap to preview)
|
||||
|
||||
**Inline images:** For HTML messages, replace `cid:<content-id>` URLs with the downloaded image data. If the inline image isn't cached, show a placeholder with a download button.
|
||||
|
||||
### Compose — Attachment Chips
|
||||
|
||||
Below the body TextEditor, attached files show as removable chips:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ 📎 report.pdf (2.1 MB) ✕ 📎 data.csv (15 KB) ✕ │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Tap ✕ to remove. The "Attach" button in the toolbar opens `.fileImporter`.
|
||||
|
||||
### Message List — Paperclip Icon
|
||||
|
||||
Thread rows show a 📎 icon when any message in the thread has attachments. Uses the existing `hasAttachments` field on `MessageSummary`.
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### MIMEParser Tests
|
||||
|
||||
- Parse single-part text/plain message → correct body, no attachments
|
||||
- Parse multipart/mixed with text + PDF → text body extracted, one attachment with correct metadata
|
||||
- Parse multipart/alternative (text + html) → both bodies extracted, no attachments
|
||||
- Parse nested multipart/mixed containing multipart/alternative → correct body + attachments
|
||||
- Base64 decode → correct binary output
|
||||
- Quoted-printable decode → correct text with non-ASCII
|
||||
- Extract filename from Content-Disposition and Content-Type name parameter
|
||||
- Section path assignment (1, 1.1, 1.2, 2)
|
||||
- Handle missing Content-Disposition (infer attachment from non-text content type)
|
||||
- Handle malformed MIME (missing boundary, truncated parts) gracefully
|
||||
|
||||
### MessageFormatter Tests (extended)
|
||||
|
||||
- Format with attachments → valid multipart/mixed output
|
||||
- Format without attachments → single-part text/plain (no multipart wrapper)
|
||||
- Base64 encoding wraps at 76 characters
|
||||
- Boundary string is unique and doesn't appear in content
|
||||
- Multiple attachments produce correct number of parts
|
||||
- Attachment content-type and filename headers correct
|
||||
|
||||
### IMAP IDLE Tests
|
||||
|
||||
- MockIMAPClient extended with IDLE simulation
|
||||
- Start monitoring triggers IDLE command
|
||||
- EXISTS response triggers onNewMail callback
|
||||
- 29-minute re-IDLE timer fires correctly
|
||||
- Connection drop triggers reconnect
|
||||
- Server without IDLE capability → monitoring not started, returns gracefully
|
||||
- Stop monitoring sends DONE and disconnects
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Sync with MIME message populates attachment table
|
||||
- Attachment download fetches correct section and caches
|
||||
- Compose with attachment produces valid multipart message via SMTP
|
||||
- IDLE notification triggers sync cycle
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
No new external dependencies. Uses existing packages + Foundation's `UTType` for MIME type detection.
|
||||
|
||||
| Package | Purpose |
|
||||
|---------|---------|
|
||||
| swift-nio-imap | IMAP protocol including IDLE (existing) |
|
||||
| swift-nio | Networking (existing) |
|
||||
| swift-nio-ssl | TLS (existing) |
|
||||
| GRDB.swift | SQLite (existing) |
|
||||
| UniformTypeIdentifiers | MIME type detection (Apple framework, no package needed) |
|
||||
|
||||
---
|
||||
|
||||
## Schema Changes
|
||||
|
||||
### Migration v4_attachmentSection
|
||||
|
||||
```sql
|
||||
ALTER TABLE attachment ADD COLUMN sectionPath TEXT;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## v0.5 Scope
|
||||
|
||||
### In
|
||||
|
||||
- MIMEParser module (multipart parsing, content decoding, section paths)
|
||||
- Attachment receiving: MIME-aware sync, attachment table population, on-demand download, cache
|
||||
- Attachment sending: file picker in compose, multipart/mixed message formatting, base64 encoding
|
||||
- Attachment UI: attachment strip in thread detail, inline image rendering, compose chips, paperclip icon
|
||||
- IMAP IDLE: dedicated connection, EXISTS notification, auto-sync trigger, 29-min re-IDLE, reconnect
|
||||
- Periodic sync interval bumped to 15 min (IDLE is the fast path)
|
||||
- Schema migration for attachment sectionPath
|
||||
|
||||
### Out (Deferred)
|
||||
|
||||
- Delegation / waiting loop (v0.5+)
|
||||
- Tags / GTD contexts (v0.5+)
|
||||
- Defer to time-of-day / notifications (v0.5+)
|
||||
- Task search in FTS5 (v0.5+)
|
||||
- CalDAV sync (v0.6)
|
||||
- Task dependencies (v0.6)
|
||||
- Rich text compose (future)
|
||||
- Multiple accounts (future)
|
||||
- Drag-and-drop attachments (future)
|
||||
- Attachment size limits / compression (future)
|
||||
Reference in New Issue
Block a user