Files

Felix Förtsch 18e7ff2c47 fix v0.5 spec: address 14 review issues (MIME, IDLE handler, hasAttachments, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-14 10:00:57 +01:00

22 KiB

Raw Blame History

Magnum Opus v0.5 — Attachments & IMAP IDLE

Goal: Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE. No more polling delay for new mail.

Builds on: v0.4 GTD email client (IMAP sync, SMTP compose, triage, tasks, ActionQueue, VTODO files).

Core Changes

v0.5 adds three capabilities:

MIMEParser — parse multipart MIME messages to extract attachments from received emails
Attachment sending — extend compose/SMTP to build multipart/mixed messages with file attachments
IMAP IDLE — persistent connection to INBOX for instant new-mail notifications

Architecture Overview

┌──────────────────────────────────────────────────────────────┐
│                      SwiftUI Apps                             │
│                                                               │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐       │
│  │ ComposeView  │  │ AttachStrip  │  │ThreadDetailView│       │
│  │ +file picker │  │ (download/   │  │ +inline images│       │
│  └──────┬───────┘  │  preview)    │  └───────┬───────┘       │
│         │          └──────┬───────┘          │               │
│         └─────────────────┼──────────────────┘               │
│                     ┌─────┴──────┐                            │
│                     │ ViewModels │                            │
│                     └─────┬──────┘                            │
├───────────────────────────┼──────────────────────────────────┤
│              MagnumOpusCore (Swift Package)                    │
│                     ┌─────┴──────┐                            │
│                     │ SyncEngine │                            │
│                     │            │                            │
│         ┌───────────┼─────────┐  │                            │
│         │   ActionQueue      │  │                            │
│         └──┬─────────┬───────┘  │                            │
│   ┌────────┘         └───────┐  │                            │
│   │ SMTPClient │  │IMAPClient│  │  ┌──────────┐              │
│   │(multipart) │  │          │  │  │ MailStore │              │
│   └────────────┘  │          │  │  │ (GRDB)   │              │
│                   │          │  │  └──────────┘              │
│                   └──────────┘  │                            │
│                   ┌──────────┐  │  ┌──────────┐              │
│                   │IMAPIdle  │  │  │MIMEParser│              │
│                   │Client    │  │  │          │              │
│                   │(NEW)     │  │  │(NEW)     │              │
│                   └──────────┘  │  └──────────┘              │
└───────────────────┼─────────────┼────────────────────────────┘
                    │             │
          ┌─────────┘             └──────────┐
          ▼                                  ▼
    ┌──────────┐                       ┌──────────┐
    │   IMAP   │ ← IDLE connection     │   SMTP   │
    │  Server  │ + sync connection     │  Server  │
    └──────────┘                       └──────────┘

MIMEParser Module

New module in MagnumOpusCore. Parses RFC 2045/2046 MIME messages. No external dependencies — MIME is line-based.

Public API

public enum MIMEParser {
    /// Parse a raw MIME message into a structured tree of parts
    public static func parse(_ rawMessage: String) -> MIMEMessage

    /// Decode content based on Content-Transfer-Encoding
    public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data

    /// Generate a unique MIME boundary string
    public static func generateBoundary() -> String
}

public struct MIMEMessage: Sendable {
    public var headers: [String: String]
    public var parts: [MIMEPart]
    public var textBody: String?       // first text/plain part
    public var htmlBody: String?       // first text/html part
    public var attachments: [MIMEAttachment]  // non-inline parts
    public var inlineImages: [MIMEAttachment] // Content-Disposition: inline
}

public struct MIMEPart: Sendable {
    public var headers: [String: String]
    public var contentType: String       // e.g., "text/plain"
    public var charset: String?          // e.g., "utf-8"
    public var transferEncoding: TransferEncoding
    public var disposition: ContentDisposition?
    public var filename: String?
    public var contentId: String?        // for inline images (cid: references)
    public var body: Data                // raw content (decoded from transfer encoding)
    public var subparts: [MIMEPart]      // for nested multipart
}

public struct MIMEAttachment: Sendable {
    public var filename: String
    public var mimeType: String
    public var size: Int                 // estimated decoded size (base64 length * 3/4)
    public var contentId: String?        // for inline images
    public var sectionPath: String       // IMAP section number (e.g., "1.2")
    public var isInline: Bool
}

public enum TransferEncoding: String, Sendable {
    case base64
    case quotedPrintable = "quoted-printable"
    case sevenBit = "7bit"
    case eightBit = "8bit"
    case binary
}

public enum ContentDisposition: Sendable {
    case inline
    case attachment
}

Parsing Logic

Parse top-level headers (split on first empty line)
Check Content-Type for multipart/* — if so, extract boundary and split body on boundary markers
Recursively parse each part
For multipart/alternative: pick text/plain and text/html parts for body
For multipart/mixed: first text part (or multipart/alternative) is body, rest are attachments
For multipart/related: HTML body + inline images (Content-ID references for cid: URLs). This is the standard container for HTML newsletters with embedded images.
For nested structures (e.g., multipart/mixed containing multipart/related containing multipart/alternative): recurse
Extract filename from Content-Disposition: attachment; filename="..." or Content-Type: ...; name="...". Decode RFC 2047 encoded words (=?charset?encoding?text?=) for non-ASCII filenames.
Estimate decoded size from base64 content length (base64 inflates ~33%): encodedLength * 3 / 4
Assign IMAP section paths based on part position (1, 1.1, 1.2, 2, etc.)

Boundary Generation

MIMEParser.generateBoundary() returns "=_MagnumOpus_\(UUID().uuidString)". The =_ prefix makes collision with base64 content essentially impossible (base64 alphabet doesn't include =_ at line starts).

RFC 2047 Decoding

Filenames may be encoded as =?utf-8?B?<base64>?= or =?utf-8?Q?<quoted-printable>?=. The parser detects this pattern and decodes to produce human-readable filenames.

Content Decoding

base64: standard Foundation Data(base64Encoded:)
quoted-printable: reuse existing quotedPrintableEncode logic from MessageFormatter, add matching decode
7bit/8bit: pass through as UTF-8

Attachment Receiving

IMAP Sync Integration

The existing IMAPClient.fetchBody(uid:) fetches only body text (BODY[TEXT]), which strips headers needed for MIME parsing. Replace with a new method that fetches the full RFC822 message:

func fetchFullMessage(uid: Int) async throws -> String

This uses BODY.PEEK[] (full message including headers) instead of BODY[TEXT]. The existing fetchBody(uid:) remains available for non-MIME use cases.

During body fetch in SyncCoordinator, replace the current raw body storage with MIME-aware parsing:

Fetch full message via IMAPClient.fetchFullMessage(uid:)
Parse with MIMEParser.parse(rawMessage)
Store textBody → message.bodyText, htmlBody → message.bodyHtml
For each attachment in MIMEMessage.attachments + inlineImages:
- Create AttachmentRecord with filename, mimeType, size, contentId, sectionPath
- Insert into attachment table (existing table, already has the right columns)
Update message.hasAttachments (see migration below)

Note on fetchEnvelopes: The existing fetchEnvelopes method also fetches body text inline during initial sync. This path should also be updated to use MIME-aware parsing, or body fetching should be deferred to the separate prefetchBodies phase (which already exists). The simpler approach: keep fetchEnvelopes fetching only envelopes (no body), and let prefetchBodies do the full RFC822 fetch with MIME parsing. This avoids doubling the bandwidth during initial sync.

IMAPClient Extension

New method for on-demand attachment download:

func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data

Uses IMAP BODY[section] fetch to retrieve a specific MIME part by section number. Returns decoded content (base64 decoded).

Attachment Storage

Downloaded attachments cached at: ~/Library/Application Support/MagnumOpus/<accountId>/attachments/<attachmentId>.<ext>

The attachment.cachePath column (existing, currently unused) is updated when content is downloaded.

Schema Changes

No new tables needed. The existing attachment table has all required columns:

id, messageId, filename, mimeType, size, contentId, cachePath

Add one column:

ALTER TABLE attachment ADD COLUMN sectionPath TEXT;

This stores the IMAP section number for on-demand fetch.

Attachment Download Flow

User taps attachment chip → check cachePath
If cached: open with QuickLook
If not cached:
- Show download progress
- Check UID validity: compare stored mailbox.uidValidity with server. If changed, UIDs are stale — show error "Message may have moved, please re-sync"
- Call IMAPClient.fetchSection(uid:mailbox:section:) with the stored sectionPath
- Save to cache directory
- Update attachment.cachePath
- Open with QuickLook

Attachment Sending

Compose Flow Changes

ComposeViewModel additions:

attachments: [(url: URL, filename: String, mimeType: String, data: Data)]
attachFile() — triggered by file picker, reads file data, adds to list. Refuses files > 25 MB with user-facing error (the SMTP server will ultimately enforce limits, but loading a 500 MB video into memory crashes the app).
removeAttachment(at:) — remove from list
totalAttachmentSize: Int — computed sum, shown in UI

ComposeView additions:

"Attach" toolbar button + ⌘⇧A shortcut → .fileImporter system file picker
Attachment chips below the body editor: filename + size + remove button
Allow multiple file selection

MessageFormatter Extension

When attachments are present, build a multipart/mixed message instead of single-part:

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----MagnumOpus-<uuid>"

------MagnumOpus-<uuid>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<body text, quoted-printable encoded>
------MagnumOpus-<uuid>
Content-Type: application/pdf; name="report.pdf"
Content-Disposition: attachment; filename="report.pdf"
Content-Transfer-Encoding: base64

<base64 encoded file content, wrapped at 76 chars>
------MagnumOpus-<uuid>--

When no attachments, keep the current simple single-part format unchanged.

The existing MessageFormatter.format(_:) is updated to accept an optional attachments parameter. If attachments are present, it delegates to formatMultipart; otherwise it produces the existing single-part output. The ActionQueue/SMTP send path calls format() as before — no changes needed downstream.

New public methods:

public static func formatMultipart(
    _ message: OutgoingMessage,
    attachments: [(filename: String, mimeType: String, data: Data)]
) -> String

public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String

MIME Type Detection

Use UTType (UniformTypeIdentifiers framework) to detect MIME type from file extension. Fallback: application/octet-stream.

IMAP IDLE

IMAPIdleClient

New actor in the IMAPClient module. Manages a dedicated, long-lived IMAP connection for IDLE on INBOX.

public actor IMAPIdleClient: Sendable {
    public init(host: String, port: Int, credentials: Credentials)

    /// Start monitoring INBOX. Calls onNewMail when server sends EXISTS.
    public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws

    /// Stop monitoring and disconnect.
    public func stopMonitoring() async
}

IDLE Protocol Sequence

→ LOGIN (authenticate)
→ SELECT INBOX
→ IDLE
← + idling
   ... server holds connection ...
← * 42 EXISTS              ← new mail arrived
→ DONE                     ← break IDLE
← OK IDLE terminated
→ [trigger sync]
→ IDLE                     ← re-enter IDLE
   ... repeat ...

Lifecycle Management

Start: After successful first sync, if server advertises IDLE capability
Re-IDLE timer: Send DONE + re-issue IDLE every 29 minutes (RFC 2177)
Reconnect: On connection drop, reconnect with exponential backoff (5s, 10s, 30s, 1m, 5m cap)
Stop: On app background (iOS), window close (macOS), or explicit stop
Fallback: If server doesn't support IDLE, keep existing periodic poll (bumped to 15 min)

SyncCoordinator Integration

// New properties
private var idleClient: IMAPIdleClient?
private var idleTask: Task<Void, Never>?

// New methods
public func startIdleMonitoring()    // check capabilities, start if supported
public func stopIdleMonitoring()     // stop IDLE, cancel task

// In startIdleMonitoring:
// 1. Check server capabilities for "IDLE"
// 2. Create IMAPIdleClient with same host/port/credentials
// 3. Call startMonitoring(onNewMail: { self.syncNow() })
// 4. Adjust periodic sync interval to 15 minutes (fallback)

NIO Implementation

IMAPIdleClient uses the same NIO patterns as IMAPClient:

IMAPConnection for the TLS channel (reuse existing class)
IMAPCommandRunner for LOGIN/SELECT (reuse)
New: IMAPIdleHandler — a separate ChannelInboundHandler that replaces the standard IMAPResponseHandler after entering IDLE mode. Unlike the standard handler (which holds a single CheckedContinuation resolved on tagged responses), IMAPIdleHandler uses an AsyncStream<IMAPIdleEvent> to deliver a stream of untagged responses. The IDLE loop consumes this stream:

enum IMAPIdleEvent: Sendable {
    case exists(Int)      // * <n> EXISTS — new mail
    case expunge(Int)     // * <n> EXPUNGE — message removed
    case idleTerminated   // OK IDLE terminated (after DONE)
}

The IDLE loop in IMAPIdleClient:

Send IDLE command
Consume AsyncStream<IMAPIdleEvent> in a for-await loop
On .exists: call onNewMail(), send DONE, wait for .idleTerminated, re-enter IDLE
On connection drop: the stream ends, trigger reconnect

This avoids the single-continuation limitation of the standard response handler.

The IDLE connection is separate from the sync connection — two concurrent connections to the IMAP server.

INBOX Resolution

The mailbox name for IDLE is resolved via MailStore.mailboxWithRole("inbox", accountId:) (using the role detection from v0.3). If no inbox role is found, falls back to the literal string "INBOX" (RFC 3501 requires this name).

Capability Check

IDLE capability is checked via the sync IMAPClient (which caches capabilities from the initial connection). startIdleMonitoring() reads this cached value — it does not require the IDLE connection to query capabilities separately.

iOS Background Behavior

IMAP IDLE is foreground-only on iOS. When the app enters background, stopIdleMonitoring() is called. On foreground resume, a fresh syncNow() is triggered to catch any mail that arrived while backgrounded, then IDLE monitoring restarts. On macOS, IDLE runs continuously while the app window is open.

Attachment UI

Thread Detail — Attachment Strip

Below each message body that has attachments, show a horizontal strip of attachment chips:

┌──────────────────────────────────────────────┐
│  📎 report.pdf (2.1 MB)  📎 photo.jpg (450 KB) │
│  [tap to download]       [tap to preview]     │
└──────────────────────────────────────────────┘

Each chip shows:

File type icon (SF Symbol based on MIME type: doc.text, photo, film, music.note, doc)
Filename
Size
Download state: not downloaded → downloading (progress) → cached (tap to preview)

Inline images: For HTML messages, replace cid:<content-id> URLs with the downloaded image data. If the inline image isn't cached, show a placeholder with a download button.

Compose — Attachment Chips

Below the body TextEditor, attached files show as removable chips:

┌──────────────────────────────────────────────┐
│  📎 report.pdf (2.1 MB) ✕   📎 data.csv (15 KB) ✕ │
└──────────────────────────────────────────────┘

Tap ✕ to remove. The "Attach" button in the toolbar opens .fileImporter.

Message List — Paperclip Icon

Thread rows show a 📎 icon when any message in the thread has attachments. Uses the existing hasAttachments field on MessageSummary.

Testing Strategy

MIMEParser Tests

Parse single-part text/plain message → correct body, no attachments
Parse multipart/mixed with text + PDF → text body extracted, one attachment with correct metadata
Parse multipart/alternative (text + html) → both bodies extracted, no attachments
Parse nested multipart/mixed containing multipart/alternative → correct body + attachments
Base64 decode → correct binary output
Quoted-printable decode → correct text with non-ASCII
Extract filename from Content-Disposition and Content-Type name parameter
Section path assignment (1, 1.1, 1.2, 2)
Handle missing Content-Disposition (infer attachment from non-text content type)
Handle malformed MIME (missing boundary, truncated parts) gracefully
Parse multipart/related (HTML + inline images with cid: references)
Decode RFC 2047 encoded filenames (=?utf-8?B?...?= and =?utf-8?Q?...?=)

MessageFormatter Tests (extended)

Format with attachments → valid multipart/mixed output
Format without attachments → single-part text/plain (no multipart wrapper)
Base64 encoding wraps at 76 characters
Boundary string is unique and doesn't appear in content
Multiple attachments produce correct number of parts
Attachment content-type and filename headers correct

IMAP IDLE Tests

MockIMAPClient extended with IDLE simulation
Start monitoring triggers IDLE command
EXISTS response triggers onNewMail callback
29-minute re-IDLE timer fires correctly
Connection drop triggers reconnect
Server without IDLE capability → monitoring not started, returns gracefully
Stop monitoring sends DONE and disconnects

Integration Tests

Sync with MIME message populates attachment table
Attachment download fetches correct section and caches
Compose with attachment produces valid multipart message via SMTP
IDLE notification triggers sync cycle

Dependencies

No new external dependencies. Uses existing packages + Foundation's UTType for MIME type detection.

Package	Purpose
swift-nio-imap	IMAP protocol including IDLE (existing)
swift-nio	Networking (existing)
swift-nio-ssl	TLS (existing)
GRDB.swift	SQLite (existing)
UniformTypeIdentifiers	MIME type detection (Apple framework, no package needed)

Schema Changes

Migration v4_attachment: Attachment MIME support

ALTER TABLE attachment ADD COLUMN sectionPath TEXT;
ALTER TABLE message ADD COLUMN hasAttachments INTEGER NOT NULL DEFAULT 0;

The hasAttachments column is populated during MIME parsing. The sectionPath stores the IMAP section number for on-demand part fetch.

Note: v4 follows after v3 migrations from v0.4 (v3_task, v3_label, v3_deferral).

v0.5 Scope

In

MIMEParser module (multipart parsing, content decoding, section paths)
Attachment receiving: MIME-aware sync, attachment table population, on-demand download, cache
Attachment sending: file picker in compose, multipart/mixed message formatting, base64 encoding
Attachment UI: attachment strip in thread detail, inline image rendering, compose chips, paperclip icon
IMAP IDLE: dedicated connection, EXISTS notification, auto-sync trigger, 29-min re-IDLE, reconnect
Periodic sync interval bumped to 15 min (IDLE is the fast path)
Schema migration for attachment sectionPath + message hasAttachments
RFC 2047 filename decoding
Per-file attachment size guard (25 MB)

Out (Deferred)

Delegation / waiting loop (v0.5+)
Tags / GTD contexts (v0.5+)
Defer to time-of-day / notifications (v0.5+)
Task search in FTS5 (v0.5+)
CalDAV sync (v0.6)
Task dependencies (v0.6)
Rich text compose (future)
Multiple accounts (future)
Drag-and-drop attachments (future)
Attachment compression (future)

22 KiB Raw Blame History

Magnum Opus v0.5 — Attachments & IMAP IDLE

Core Changes

Architecture Overview

MIMEParser Module

Public API

Parsing Logic

Boundary Generation

RFC 2047 Decoding

Content Decoding

Attachment Receiving

IMAP Sync Integration

IMAPClient Extension

Attachment Storage

Schema Changes

Attachment Download Flow

Attachment Sending

Compose Flow Changes

MessageFormatter Extension

MIME Type Detection

IMAP IDLE

IMAPIdleClient

IDLE Protocol Sequence

Lifecycle Management

SyncCoordinator Integration

NIO Implementation

INBOX Resolution

Capability Check

iOS Background Behavior

Attachment UI

Thread Detail — Attachment Strip

Compose — Attachment Chips

Message List — Paperclip Icon

Testing Strategy

MIMEParser Tests

MessageFormatter Tests (extended)

IMAP IDLE Tests

Integration Tests

Dependencies

Schema Changes

Migration v4_attachment: Attachment MIME support

v0.5 Scope

In

Out (Deferred)

22 KiB

Raw Blame History