# Magnum Opus v0.5 — Attachments & IMAP IDLE

**Goal:** Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE. No more polling delay for new mail.

**Builds on:** v0.4 GTD email client (IMAP sync, SMTP compose, triage, tasks, ActionQueue, VTODO files).

---

## Core Changes

v0.5 adds three capabilities:

1. **MIMEParser** — parse multipart MIME messages to extract attachments from received emails
2. **Attachment sending** — extend compose/SMTP to build multipart/mixed messages with file attachments
3. **IMAP IDLE** — persistent connection to INBOX for instant new-mail notifications

---

## Architecture Overview

```
┌──────────────────────────────────────────────────────────────┐
│                      SwiftUI Apps                             │
│                                                               │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐       │
│  │ ComposeView  │  │ AttachStrip  │  │ThreadDetailView│       │
│  │ +file picker │  │ (download/   │  │ +inline images│       │
│  └──────┬───────┘  │  preview)    │  └───────┬───────┘       │
│         │          └──────┬───────┘          │               │
│         └─────────────────┼──────────────────┘               │
│                     ┌─────┴──────┐                            │
│                     │ ViewModels │                            │
│                     └─────┬──────┘                            │
├───────────────────────────┼──────────────────────────────────┤
│              MagnumOpusCore (Swift Package)                    │
│                     ┌─────┴──────┐                            │
│                     │ SyncEngine │                            │
│                     │            │                            │
│         ┌───────────┼─────────┐  │                            │
│         │   ActionQueue      │  │                            │
│         └──┬─────────┬───────┘  │                            │
│   ┌────────┘         └───────┐  │                            │
│   │ SMTPClient │  │IMAPClient│  │  ┌──────────┐              │
│   │(multipart) │  │          │  │  │ MailStore │              │
│   └────────────┘  │          │  │  │ (GRDB)   │              │
│                   │          │  │  └──────────┘              │
│                   └──────────┘  │                            │
│                   ┌──────────┐  │  ┌──────────┐              │
│                   │IMAPIdle  │  │  │MIMEParser│              │
│                   │Client    │  │  │          │              │
│                   │(NEW)     │  │  │(NEW)     │              │
│                   └──────────┘  │  └──────────┘              │
└───────────────────┼─────────────┼────────────────────────────┘
                    │             │
          ┌─────────┘             └──────────┐
          ▼                                  ▼
    ┌──────────┐                       ┌──────────┐
    │   IMAP   │ ← IDLE connection     │   SMTP   │
    │  Server  │ + sync connection     │  Server  │
    └──────────┘                       └──────────┘
```

---

## MIMEParser Module

New module in MagnumOpusCore. Parses RFC 2045/2046 MIME messages. No external dependencies — MIME is line-based.

### Public API

```swift
public enum MIMEParser {
    /// Parse a raw MIME message into a structured tree of parts
    public static func parse(_ rawMessage: String) -> MIMEMessage

    /// Decode content based on Content-Transfer-Encoding
    public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data

    /// Generate a unique MIME boundary string
    public static func generateBoundary() -> String
}

public struct MIMEMessage: Sendable {
    public var headers: [String: String]
    public var parts: [MIMEPart]
    public var textBody: String?       // first text/plain part
    public var htmlBody: String?       // first text/html part
    public var attachments: [MIMEAttachment]  // non-inline parts
    public var inlineImages: [MIMEAttachment] // Content-Disposition: inline
}

public struct MIMEPart: Sendable {
    public var headers: [String: String]
    public var contentType: String       // e.g., "text/plain"
    public var charset: String?          // e.g., "utf-8"
    public var transferEncoding: TransferEncoding
    public var disposition: ContentDisposition?
    public var filename: String?
    public var contentId: String?        // for inline images (cid: references)
    public var body: Data                // raw content (decoded from transfer encoding)
    public var subparts: [MIMEPart]      // for nested multipart
}

public struct MIMEAttachment: Sendable {
    public var filename: String
    public var mimeType: String
    public var size: Int                 // estimated decoded size (base64 length * 3/4)
    public var contentId: String?        // for inline images
    public var sectionPath: String       // IMAP section number (e.g., "1.2")
    public var isInline: Bool
}

public enum TransferEncoding: String, Sendable {
    case base64
    case quotedPrintable = "quoted-printable"
    case sevenBit = "7bit"
    case eightBit = "8bit"
    case binary
}

public enum ContentDisposition: Sendable {
    case inline
    case attachment
}
```

### Parsing Logic

1. Parse top-level headers (split on first empty line)
2. Check `Content-Type` for `multipart/*` — if so, extract boundary and split body on boundary markers
3. Recursively parse each part
4. For `multipart/alternative`: pick text/plain and text/html parts for body
5. For `multipart/mixed`: first text part (or multipart/alternative) is body, rest are attachments
6. For `multipart/related`: HTML body + inline images (Content-ID references for `cid:` URLs). This is the standard container for HTML newsletters with embedded images.
7. For nested structures (e.g., `multipart/mixed` containing `multipart/related` containing `multipart/alternative`): recurse
8. Extract `filename` from `Content-Disposition: attachment; filename="..."` or `Content-Type: ...; name="..."`. Decode RFC 2047 encoded words (`=?charset?encoding?text?=`) for non-ASCII filenames.
9. Estimate decoded size from base64 content length (base64 inflates ~33%): `encodedLength * 3 / 4`
10. Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.)

### Boundary Generation

`MIMEParser.generateBoundary()` returns `"=_MagnumOpus_\(UUID().uuidString)"`. The `=_` prefix makes collision with base64 content essentially impossible (base64 alphabet doesn't include `=_` at line starts).

### RFC 2047 Decoding

Filenames may be encoded as `=?utf-8?B?<base64>?=` or `=?utf-8?Q?<quoted-printable>?=`. The parser detects this pattern and decodes to produce human-readable filenames.

### Content Decoding

- **base64:** standard Foundation `Data(base64Encoded:)`
- **quoted-printable:** reuse existing `quotedPrintableEncode` logic from MessageFormatter, add matching decode
- **7bit/8bit:** pass through as UTF-8

---

## Attachment Receiving

### IMAP Sync Integration

The existing `IMAPClient.fetchBody(uid:)` fetches only body text (BODY[TEXT]), which strips headers needed for MIME parsing. Replace with a new method that fetches the full RFC822 message:

```swift
func fetchFullMessage(uid: Int) async throws -> String
```

This uses `BODY.PEEK[]` (full message including headers) instead of `BODY[TEXT]`. The existing `fetchBody(uid:)` remains available for non-MIME use cases.

During body fetch in `SyncCoordinator`, replace the current raw body storage with MIME-aware parsing:

1. Fetch full message via `IMAPClient.fetchFullMessage(uid:)`
2. Parse with `MIMEParser.parse(rawMessage)`
3. Store `textBody` → `message.bodyText`, `htmlBody` → `message.bodyHtml`
4. For each attachment in `MIMEMessage.attachments` + `inlineImages`:
   - Create `AttachmentRecord` with filename, mimeType, size, contentId, sectionPath
   - Insert into `attachment` table (existing table, already has the right columns)
5. Update `message.hasAttachments` (see migration below)

**Note on fetchEnvelopes:** The existing `fetchEnvelopes` method also fetches body text inline during initial sync. This path should also be updated to use MIME-aware parsing, or body fetching should be deferred to the separate `prefetchBodies` phase (which already exists). The simpler approach: keep `fetchEnvelopes` fetching only envelopes (no body), and let `prefetchBodies` do the full RFC822 fetch with MIME parsing. This avoids doubling the bandwidth during initial sync.

### IMAPClient Extension

New method for on-demand attachment download:

```swift
func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data
```

Uses IMAP `BODY[section]` fetch to retrieve a specific MIME part by section number. Returns decoded content (base64 decoded).

### Attachment Storage

Downloaded attachments cached at:
`~/Library/Application Support/MagnumOpus/<accountId>/attachments/<attachmentId>.<ext>`

The `attachment.cachePath` column (existing, currently unused) is updated when content is downloaded.

### Schema Changes

No new tables needed. The existing `attachment` table has all required columns:
- id, messageId, filename, mimeType, size, contentId, cachePath

Add one column:

```sql
ALTER TABLE attachment ADD COLUMN sectionPath TEXT;
```

This stores the IMAP section number for on-demand fetch.

### Attachment Download Flow

1. User taps attachment chip → check `cachePath`
2. If cached: open with QuickLook
3. If not cached:
   - Show download progress
   - Check UID validity: compare stored `mailbox.uidValidity` with server. If changed, UIDs are stale — show error "Message may have moved, please re-sync"
   - Call `IMAPClient.fetchSection(uid:mailbox:section:)` with the stored `sectionPath`
   - Save to cache directory
   - Update `attachment.cachePath`
   - Open with QuickLook

---

## Attachment Sending

### Compose Flow Changes

**ComposeViewModel additions:**
- `attachments: [(url: URL, filename: String, mimeType: String, data: Data)]`
- `attachFile()` — triggered by file picker, reads file data, adds to list. Refuses files > 25 MB with user-facing error (the SMTP server will ultimately enforce limits, but loading a 500 MB video into memory crashes the app).
- `removeAttachment(at:)` — remove from list
- `totalAttachmentSize: Int` — computed sum, shown in UI

**ComposeView additions:**
- "Attach" toolbar button + `⌘⇧A` shortcut → `.fileImporter` system file picker
- Attachment chips below the body editor: filename + size + remove button
- Allow multiple file selection

### MessageFormatter Extension

When attachments are present, build a `multipart/mixed` message instead of single-part:

```
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----MagnumOpus-<uuid>"

------MagnumOpus-<uuid>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<body text, quoted-printable encoded>
------MagnumOpus-<uuid>
Content-Type: application/pdf; name="report.pdf"
Content-Disposition: attachment; filename="report.pdf"
Content-Transfer-Encoding: base64

<base64 encoded file content, wrapped at 76 chars>
------MagnumOpus-<uuid>--
```

When no attachments, keep the current simple single-part format unchanged.

The existing `MessageFormatter.format(_:)` is updated to accept an optional attachments parameter. If attachments are present, it delegates to `formatMultipart`; otherwise it produces the existing single-part output. The ActionQueue/SMTP send path calls `format()` as before — no changes needed downstream.

New public methods:

```swift
public static func formatMultipart(
    _ message: OutgoingMessage,
    attachments: [(filename: String, mimeType: String, data: Data)]
) -> String

public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String
```

### MIME Type Detection

Use `UTType` (UniformTypeIdentifiers framework) to detect MIME type from file extension. Fallback: `application/octet-stream`.

---

## IMAP IDLE

### IMAPIdleClient

New actor in the IMAPClient module. Manages a dedicated, long-lived IMAP connection for IDLE on INBOX.

```swift
public actor IMAPIdleClient: Sendable {
    public init(host: String, port: Int, credentials: Credentials)

    /// Start monitoring INBOX. Calls onNewMail when server sends EXISTS.
    public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws

    /// Stop monitoring and disconnect.
    public func stopMonitoring() async
}
```

### IDLE Protocol Sequence

```
→ LOGIN (authenticate)
→ SELECT INBOX
→ IDLE
← + idling
   ... server holds connection ...
← * 42 EXISTS              ← new mail arrived
→ DONE                     ← break IDLE
← OK IDLE terminated
→ [trigger sync]
→ IDLE                     ← re-enter IDLE
   ... repeat ...
```

### Lifecycle Management

- **Start:** After successful first sync, if server advertises `IDLE` capability
- **Re-IDLE timer:** Send `DONE` + re-issue `IDLE` every 29 minutes (RFC 2177)
- **Reconnect:** On connection drop, reconnect with exponential backoff (5s, 10s, 30s, 1m, 5m cap)
- **Stop:** On app background (iOS), window close (macOS), or explicit stop
- **Fallback:** If server doesn't support IDLE, keep existing periodic poll (bumped to 15 min)

### SyncCoordinator Integration

```swift
// New properties
private var idleClient: IMAPIdleClient?
private var idleTask: Task<Void, Never>?

// New methods
public func startIdleMonitoring()    // check capabilities, start if supported
public func stopIdleMonitoring()     // stop IDLE, cancel task

// In startIdleMonitoring:
// 1. Check server capabilities for "IDLE"
// 2. Create IMAPIdleClient with same host/port/credentials
// 3. Call startMonitoring(onNewMail: { self.syncNow() })
// 4. Adjust periodic sync interval to 15 minutes (fallback)
```

### NIO Implementation

`IMAPIdleClient` uses the same NIO patterns as `IMAPClient`:
- `IMAPConnection` for the TLS channel (reuse existing class)
- `IMAPCommandRunner` for LOGIN/SELECT (reuse)
- **New: `IMAPIdleHandler`** — a separate `ChannelInboundHandler` that replaces the standard `IMAPResponseHandler` after entering IDLE mode. Unlike the standard handler (which holds a single `CheckedContinuation` resolved on tagged responses), `IMAPIdleHandler` uses an `AsyncStream<IMAPIdleEvent>` to deliver a stream of untagged responses. The IDLE loop consumes this stream:

```swift
enum IMAPIdleEvent: Sendable {
    case exists(Int)      // * <n> EXISTS — new mail
    case expunge(Int)     // * <n> EXPUNGE — message removed
    case idleTerminated   // OK IDLE terminated (after DONE)
}
```

The IDLE loop in `IMAPIdleClient`:
1. Send `IDLE` command
2. Consume `AsyncStream<IMAPIdleEvent>` in a for-await loop
3. On `.exists`: call `onNewMail()`, send `DONE`, wait for `.idleTerminated`, re-enter IDLE
4. On connection drop: the stream ends, trigger reconnect

This avoids the single-continuation limitation of the standard response handler.

The IDLE connection is separate from the sync connection — two concurrent connections to the IMAP server.

### INBOX Resolution

The mailbox name for IDLE is resolved via `MailStore.mailboxWithRole("inbox", accountId:)` (using the role detection from v0.3). If no inbox role is found, falls back to the literal string "INBOX" (RFC 3501 requires this name).

### Capability Check

IDLE capability is checked via the sync `IMAPClient` (which caches capabilities from the initial connection). `startIdleMonitoring()` reads this cached value — it does not require the IDLE connection to query capabilities separately.

### iOS Background Behavior

IMAP IDLE is **foreground-only on iOS**. When the app enters background, `stopIdleMonitoring()` is called. On foreground resume, a fresh `syncNow()` is triggered to catch any mail that arrived while backgrounded, then IDLE monitoring restarts. On macOS, IDLE runs continuously while the app window is open.

---

## Attachment UI

### Thread Detail — Attachment Strip

Below each message body that has attachments, show a horizontal strip of attachment chips:

```
┌──────────────────────────────────────────────┐
│  📎 report.pdf (2.1 MB)  📎 photo.jpg (450 KB) │
│  [tap to download]       [tap to preview]     │
└──────────────────────────────────────────────┘
```

Each chip shows:
- File type icon (SF Symbol based on MIME type: doc.text, photo, film, music.note, doc)
- Filename
- Size
- Download state: not downloaded → downloading (progress) → cached (tap to preview)

**Inline images:** For HTML messages, replace `cid:<content-id>` URLs with the downloaded image data. If the inline image isn't cached, show a placeholder with a download button.

### Compose — Attachment Chips

Below the body TextEditor, attached files show as removable chips:

```
┌──────────────────────────────────────────────┐
│  📎 report.pdf (2.1 MB) ✕   📎 data.csv (15 KB) ✕ │
└──────────────────────────────────────────────┘
```

Tap ✕ to remove. The "Attach" button in the toolbar opens `.fileImporter`.

### Message List — Paperclip Icon

Thread rows show a 📎 icon when any message in the thread has attachments. Uses the existing `hasAttachments` field on `MessageSummary`.

---

## Testing Strategy

### MIMEParser Tests

- Parse single-part text/plain message → correct body, no attachments
- Parse multipart/mixed with text + PDF → text body extracted, one attachment with correct metadata
- Parse multipart/alternative (text + html) → both bodies extracted, no attachments
- Parse nested multipart/mixed containing multipart/alternative → correct body + attachments
- Base64 decode → correct binary output
- Quoted-printable decode → correct text with non-ASCII
- Extract filename from Content-Disposition and Content-Type name parameter
- Section path assignment (1, 1.1, 1.2, 2)
- Handle missing Content-Disposition (infer attachment from non-text content type)
- Handle malformed MIME (missing boundary, truncated parts) gracefully
- Parse multipart/related (HTML + inline images with cid: references)
- Decode RFC 2047 encoded filenames (=?utf-8?B?...?= and =?utf-8?Q?...?=)

### MessageFormatter Tests (extended)

- Format with attachments → valid multipart/mixed output
- Format without attachments → single-part text/plain (no multipart wrapper)
- Base64 encoding wraps at 76 characters
- Boundary string is unique and doesn't appear in content
- Multiple attachments produce correct number of parts
- Attachment content-type and filename headers correct

### IMAP IDLE Tests

- MockIMAPClient extended with IDLE simulation
- Start monitoring triggers IDLE command
- EXISTS response triggers onNewMail callback
- 29-minute re-IDLE timer fires correctly
- Connection drop triggers reconnect
- Server without IDLE capability → monitoring not started, returns gracefully
- Stop monitoring sends DONE and disconnects

### Integration Tests

- Sync with MIME message populates attachment table
- Attachment download fetches correct section and caches
- Compose with attachment produces valid multipart message via SMTP
- IDLE notification triggers sync cycle

---

## Dependencies

No new external dependencies. Uses existing packages + Foundation's `UTType` for MIME type detection.

| Package | Purpose |
|---------|---------|
| swift-nio-imap | IMAP protocol including IDLE (existing) |
| swift-nio | Networking (existing) |
| swift-nio-ssl | TLS (existing) |
| GRDB.swift | SQLite (existing) |
| UniformTypeIdentifiers | MIME type detection (Apple framework, no package needed) |

---

## Schema Changes

### Migration v4_attachment: Attachment MIME support

```sql
ALTER TABLE attachment ADD COLUMN sectionPath TEXT;
ALTER TABLE message ADD COLUMN hasAttachments INTEGER NOT NULL DEFAULT 0;
```

The `hasAttachments` column is populated during MIME parsing. The `sectionPath` stores the IMAP section number for on-demand part fetch.

Note: v4 follows after v3 migrations from v0.4 (v3_task, v3_label, v3_deferral).

---

## v0.5 Scope

### In

- MIMEParser module (multipart parsing, content decoding, section paths)
- Attachment receiving: MIME-aware sync, attachment table population, on-demand download, cache
- Attachment sending: file picker in compose, multipart/mixed message formatting, base64 encoding
- Attachment UI: attachment strip in thread detail, inline image rendering, compose chips, paperclip icon
- IMAP IDLE: dedicated connection, EXISTS notification, auto-sync trigger, 29-min re-IDLE, reconnect
- Periodic sync interval bumped to 15 min (IDLE is the fast path)
- Schema migration for attachment sectionPath + message hasAttachments
- RFC 2047 filename decoding
- Per-file attachment size guard (25 MB)

### Out (Deferred)

- Delegation / waiting loop (v0.5+)
- Tags / GTD contexts (v0.5+)
- Defer to time-of-day / notifications (v0.5+)
- Task search in FTS5 (v0.5+)
- CalDAV sync (v0.6)
- Task dependencies (v0.6)
- Rich text compose (future)
- Multiple accounts (future)
- Drag-and-drop attachments (future)
- Attachment compression (future)
