MagnumOpus/docs/plans/2026-03-13-v0.2-native-email-client-design.md

Magnum Opus v0.2 — Native Swift Email Client

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Build a read-only email client in pure Swift that runs on macOS, iOS, and iPadOS. Prove the on-device architecture: IMAP sync → SQLite cache → SwiftUI three-column UI with threaded message view and full-text search.

Architectural pivot from v0.1: No remote server. All sync, indexing, and storage run on-device. Standard protocols (IMAP) connect directly to mail servers. The v0.1 TypeScript prototype (branch feature/v0.1-backend-and-macos) remains as reference.

Design document: docs/plans/2026-03-10-magnum-opus-design.md (architecture section superseded by this document — on-device Swift replaces the Hono/Uberspace backend)

---

Architecture Overview

Everything runs on-device. The app connects directly to IMAP servers via swift-nio-imap, stores messages in SQLite via GRDB, and renders with SwiftUI. No intermediate server, no CLI tool dependencies, no file-based storage layer.

┌─────────────────────────────────────────────────┐
│                   SwiftUI Apps                   │
│         (macOS / iOS / iPadOS targets)           │
│                                                  │
│  ┌──────────┐  ┌────────────┐  ┌─────────────┐  │
│  │ Sidebar  │  │ ThreadList │  │ ThreadDetail│  │
│  └────┬─────┘  └─────┬──────┘  └──────┬──────┘  │
│       └───────────────┴────────────────┘         │
│                       │ @Observable              │
│              ┌────────┴────────┐                 │
│              │   ViewModels    │                 │
│              └────────┬────────┘                 │
├───────────────────────┼──────────────────────────┤
│           MagnumOpusCore (Swift Package)         │
│              ┌────────┴────────┐                 │
│              │  SyncEngine     │                 │
│              │  (coordinator)  │                 │
│              └───┬─────────┬───┘                 │
│         ┌────────┘         └────────┐            │
│  ┌──────┴──────┐           ┌───────┴───────┐    │
│  │  IMAPClient │           │   MailStore   │    │
│  │  (nio-imap) │           │  (GRDB+FTS5) │    │
│  └──────┬──────┘           └───────────────┘    │
│         │                                        │
└─────────┼────────────────────────────────────────┘
          │ TLS
          ▼
    ┌───────────┐
    │ IMAP      │
    │ Server    │
    └───────────┘

Data flow: SyncCoordinator triggers IMAPClient → fetches message deltas → writes to MailStore (SQLite) → GRDB ValueObservation pushes updates to @Observable ViewModels → SwiftUI re-renders.

---

Package & Project Structure

MagnumOpus/
├── Packages/
│   └── MagnumOpusCore/
│       ├── Package.swift
│       ├── Sources/
│       │   ├── Models/            ← shared types
│       │   ├── IMAPClient/        ← IMAP protocol layer
│       │   ├── MailStore/         ← SQLite + GRDB + FTS5
│       │   └── SyncEngine/        ← sync coordinator
│       └── Tests/
│           ├── IMAPClientTests/
│           ├── MailStoreTests/
│           └── SyncEngineTests/
├── Apps/
│   ├── macOS/                     ← macOS SwiftUI app
│   └── iOS/                       ← iOS/iPadOS SwiftUI app
├── docs/
├── Ideas/
└── scripts/

Dependency direction: SyncEngine → IMAPClient + MailStore → Models. App targets import SyncEngine and MailStore.

---

Data Model

Core Types (Models module)

• Account — IMAP server config (SMTP deferred to v0.3). Credentials stored in Keychain, not SQLite.
• Mailbox — IMAP folder (Inbox, Sent, Drafts, Archive, Trash, custom). Tracks uidValidity and uidNext for delta sync.
• Message — email with headers, body, attachment metadata. Identified by RFC 5322 Message-ID.
• Thread — group of messages linked by In-Reply-To / References headers.

SQLite Schema

CREATE TABLE account (
    id          TEXT PRIMARY KEY,
    name        TEXT NOT NULL,
    email       TEXT NOT NULL,
    imapHost    TEXT NOT NULL,
    imapPort    INTEGER NOT NULL
    -- SMTP fields added in v0.3
);

CREATE TABLE mailbox (
    id          TEXT PRIMARY KEY,
    accountId   TEXT NOT NULL REFERENCES account(id),
    name        TEXT NOT NULL,
    uidValidity INTEGER NOT NULL,
    uidNext     INTEGER NOT NULL
);

CREATE TABLE message (
    id          TEXT PRIMARY KEY,
    accountId   TEXT NOT NULL REFERENCES account(id),
    mailboxId   TEXT NOT NULL REFERENCES mailbox(id),
    uid         INTEGER NOT NULL,
    messageId   TEXT,
    inReplyTo   TEXT,
    refs        TEXT,
    subject     TEXT,
    fromAddress TEXT,
    fromName    TEXT,
    toAddresses TEXT,               -- JSON array of {"name", "address"} objects
    ccAddresses TEXT,               -- JSON array of {"name", "address"} objects
    date        TEXT NOT NULL,
    snippet     TEXT,
    bodyText    TEXT,
    bodyHtml    TEXT,
    isRead      INTEGER NOT NULL DEFAULT 0,
    isFlagged   INTEGER NOT NULL DEFAULT 0,
    size        INTEGER NOT NULL DEFAULT 0,
    UNIQUE(mailboxId, uid)
);

CREATE TABLE thread (
    id           TEXT PRIMARY KEY,
    accountId    TEXT NOT NULL REFERENCES account(id),
    subject      TEXT,
    lastDate     TEXT NOT NULL,
    messageCount INTEGER NOT NULL DEFAULT 0
);

CREATE TABLE threadMessage (
    threadId  TEXT NOT NULL REFERENCES thread(id),
    messageId TEXT NOT NULL REFERENCES message(id),
    PRIMARY KEY (threadId, messageId)
);

CREATE TABLE attachment (
    id          TEXT PRIMARY KEY,
    messageId   TEXT NOT NULL REFERENCES message(id),
    filename    TEXT,
    mimeType    TEXT NOT NULL,
    size        INTEGER NOT NULL DEFAULT 0,
    contentId   TEXT,               -- for inline images (CID references)
    cachePath   TEXT                -- path in app sandbox, NULL until downloaded
);

CREATE VIRTUAL TABLE messageFts USING fts5(
    subject, fromName, fromAddress, bodyText,
    content='message', content_rowid='rowid'
);
-- FTS sync: the SQL above is illustrative. Implementation must use GRDB's
-- DatabaseMigrator with db.create(virtualTable:using:) for automatic
-- trigger generation to keep messageFts in sync on insert/update/delete.

Cross-Mailbox Deduplication

The same message can appear in multiple IMAP mailboxes (e.g., CC to self → Inbox + Sent). Each IMAP copy gets its own row in the message table (different mailboxId + uid). Thread reconstruction deduplicates by messageId (RFC 5322 Message-ID): when threading, all rows sharing the same messageId join the same thread. The threadMessage join table references the message.id (UUID), so a thread may contain multiple rows for the same logical message — the UI deduplicates by messageId when rendering. Messages without a messageId (broken mailers) cannot be deduplicated and each gets its own single-message thread.

Thread Reconstruction

Simplified JWZ algorithm — no subject-based fallback grouping (produces false matches):

1. New message arrives: extract Message-ID, In-Reply-To, References
2. Look up existing threads where any member shares a reference
3. If found → add to that thread (merge if multiple threads match)
4. If not found → create new thread
5. Update thread metadata (last date, message count, subject from earliest known message in thread — stripped of Re:/Fwd: prefixes)

---

IMAP Client Layer

Complexity Note

swift-nio-imap is a low-level protocol encoder/decoder, not a ready-made IMAP client. Building the IMAPClient actor requires: TLS negotiation, IMAP state machine management, command-response pairing, untagged response handling, literal continuations, and error recovery. This is the single largest implementation effort in v0.2 and must be broken into multiple implementation tasks with explicit milestones.

Public API

actor IMAPClient {
    init(host: String, port: Int, credentials: Credentials)

    func connect() async throws
    func disconnect() async throws

    func listMailboxes() async throws -> [MailboxInfo]
    func selectMailbox(_ name: String) async throws -> MailboxStatus

    func fetchMessages(uids: UIDRange, fields: FetchFields) async throws -> [FetchedMessage]
    func fetchNew(since uid: UID) async throws -> [FetchedMessage]
    func fetchFlags(uids: UIDRange) async throws -> [UIDFlagsPair]
}

Internal Layers

The actor is built in layers:

1. Connection — SwiftNIO Channel with TLS (via swift-nio-ssl), handles connect/disconnect/reconnect
2. Command pipeline — sends IMAP commands, pairs tagged responses, handles continuation requests
3. Response parser — processes untagged responses (EXISTS, EXPUNGE, FLAGS updates)
4. High-level operations — listMailboxes, selectMailbox, fetchMessages built on the pipeline

Sync Strategy (Poll-Based for v0.2)

1. Connect to IMAP server
2. SELECT each mailbox, compare UIDVALIDITY with stored value
   • Changed → full re-sync of that mailbox (rare — means server rebuilt UIDs)
   • Same → fetch messages with UID > last known uidNext
3. Fetch envelopes + snippets eagerly; full bodies lazily
4. Fetch flag changes from server (read-only — v0.2 does not write flags back to server). Local isRead/isFlagged state mirrors the server. Opening a message locally does not change flags in v0.2 — read state is purely server-driven.
5. Disconnect, schedule next poll

Eager vs. Lazy Fetching

• Eager (on sync): envelope (from, to, subject, date, message-id, references), flags, snippet (~200 chars of body)
• Lazy (on open): full body text, HTML body, attachments — fetched when user opens message, then cached in SQLite
• Background fill: after initial sync, progressively fetch bodies for recent messages (last 30 days) so they're available offline

Attachment Caching

Attachment metadata stored eagerly in SQLite. Binary content downloaded on demand, cached to disk in the app sandbox (not SQLite — attachments can be large).

---

Sync Engine

Orchestrates the full sync lifecycle:

@Observable
final class SyncCoordinator {
    private let account: Account
    private let imapClient: IMAPClient
    private let store: MailStore

    var syncState: SyncState  // .idle, .syncing(progress), .error(Error)
    var events: AsyncStream<SyncEvent>

    // 5-minute interval applies in foreground only.
    // On iOS background, sync uses BGAppRefreshTask at system-determined intervals.
    func startPeriodicSync(interval: Duration = .seconds(300))
    func syncNow() async throws
    func stopSync()
}

enum SyncEvent {
    case newMessages(count: Int, mailbox: String)
    case flagsChanged(messageIds: [String])
    case syncStarted
    case syncCompleted
    case syncFailed(Error)
}

MailStore — Read/Write Interface

final class MailStore {
    func threads(in mailbox: String, limit: Int, offset: Int) -> [Thread]
    func messages(in thread: Thread) -> [Message]
    func message(id: String) -> Message?
    func search(query: String) -> [Message]

    func insertMessages(_ messages: [FetchedMessage]) throws
    func updateFlags(uid: UID, mailbox: String, flags: MessageFlags) throws
    func storeBody(messageId: String, text: String?, html: String?) throws
    func reconstructThreads(for messages: [Message]) throws

    func observeThreads(in mailbox: String) -> AsyncValueObservation<[Thread]>
}

GRDB ValueObservation is the sole mechanism for reactive UI updates. MailStore itself is not @Observable — it provides ValueObservation-based streams that ViewModels consume. This replaces the SSE pattern from v0.1.

Error Handling & Offline Behavior

The app must remain fully usable with cached data when offline. Error strategy:

• Connection failure / offline: SyncState moves to .error. UI shows a non-intrusive banner ("Offline — showing cached mail"). All cached data remains browsable and searchable.
• Authentication failure: Surface to user immediately with option to re-enter credentials. Do not retry automatically (avoids account lockout).
• Mid-sync connection drop: Abort current sync cycle gracefully. Already-stored messages are kept. Next poll retries from the last successful uidNext.
• Retry policy: Exponential backoff starting at 30 seconds, capped at 15 minutes. Reset on successful sync or manual trigger.
• TLS failure: Do not fall back to plaintext. Surface error to user.

---

SwiftUI App Layer

View Hierarchy

AppRoot
├── AccountSetupView            ← first launch: configure IMAP
└── MainView (NavigationSplitView)
    ├── Sidebar
    │   └── MailboxListView     ← Inbox, Sent, Drafts, Archive, Trash, custom
    ├── Content
    │   └── ThreadListView      ← threads in selected mailbox
    └── Detail
        └── ThreadDetailView    ← messages in selected thread
            └── MessageView     ← single message (header, body, attachments)

HTML Email Rendering

• Use WKWebView for HTML email display (works on macOS, iOS, iPadOS)
• Block remote content by default (tracking pixels, external images). Show a "Load remote content" button.
• For multipart/alternative messages, prefer plain text; offer a toggle to view HTML
• Strip <script> tags and event handler attributes before rendering
• Inline CSS only — no external stylesheet loading
• Attachments with Content-ID (inline images) are resolved from the local cache

Platform Adaptation

NavigationSplitView handles platform differences automatically:

• macOS: three-column layout always visible, keyboard shortcuts
• iPad landscape: three columns, collapsible sidebar
• iPad portrait / iPhone: push navigation (sidebar → thread list → detail)

Account Setup (First Launch)

1. User enters email address
2. Auto-discovery: query Mozilla ISPDB (Thunderbird autoconfig database) as primary, DNS SRV records (RFC 6186) as fallback
3. Fall back to manual IMAP host + port entry
4. Credentials stored in Keychain
5. Initial sync begins immediately after setup

SMTP configuration is deferred to v0.3 (compose/reply). v0.2 setup only collects IMAP details.

Data Flow

SyncCoordinator (background sync)
       ↓ writes
    MailStore (SQLite via GRDB)
       ↓ ValueObservation
  @Observable ViewModels
       ↓
    SwiftUI Views

No manual refresh needed. ViewModels observe two sources: MailStore (via GRDB ValueObservation) for data, and SyncCoordinator (via @Observable) for sync status (progress, errors). Background sync + GRDB observation = automatic UI updates.

---

Dependencies

Package	Purpose
swift-nio-imap	IMAP protocol layer
swift-nio	Networking (transitive)
swift-nio-ssl	TLS for IMAP/SMTP
GRDB.swift	SQLite, FTS5, ValueObservation

Four dependencies total (two transitive). No bloat.

---

Testing Strategy

• IMAPClientTests — mock IMAP server via swift-nio-imap test utilities. Verify connection, fetch, flag sync, UID tracking, error handling.
• MailStoreTests — in-memory SQLite. Test insert, query, thread reconstruction, FTS5 search, observation. Most testable layer.
• SyncEngineTests — mock IMAPClient + real MailStore (in-memory). Test full sync flow: delta detection, message storage, flag reconciliation, error recovery.
• UI tests — minimal for v0.2. Verify navigation flow on both platforms.

Not tested in v0.2: performance at scale, real IMAP server integration.

---

v0.2 Scope

In

• Single account (IMAP only, read-only — no flag write-back to server)
• IMAP sync via swift-nio-imap (poll-based, configurable interval)
• SQLite cache with GRDB (envelope eager, body lazy + cached)
• FTS5 full-text search
• Thread reconstruction (Message-ID / In-Reply-To / References)
• Three-column SwiftUI layout (macOS + iOS + iPadOS)
• Mailbox navigation (Inbox, Sent, Drafts, Archive, Trash, custom folders)
• Message reading with HTML/plain text display
• Attachment metadata display (download on demand)
• Account setup with auto-discovery (Mozilla ISPDB + DNS SRV)
• Keychain credential storage
• Background body prefetch (last 30 days)

Out (Deferred)

• Compose / reply / forward (needs SMTP client — v0.3)
• Triage actions: do / defer / delegate / file / discard (v0.3)
• Task management / VTODO (v0.4+)
• CalDAV / CardDAV sync (v0.4+)
• IMAP IDLE for real-time push (optimization, post-v0.2)
• Multiple accounts
• Keyboard shortcuts beyond system defaults
• Notifications

---

Future Evolution

v0.2 is the foundation. The path forward:

• v0.3: SMTP client (SwiftNIO-based), compose/reply/forward, basic triage (archive, delete, flag)
• v0.4: VTODO tasks via CalDAV, unified inbox mixing email + tasks, GTD triage workflow
• v0.5: Contacts via CardDAV, calendar via CalDAV, delegation workflow
• Later: IMAP IDLE, multiple accounts, keyboard-first triage, append-only history