add v0.2 native swift email client design spec
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
371
docs/plans/2026-03-13-v0.2-native-email-client-design.md
Normal file
371
docs/plans/2026-03-13-v0.2-native-email-client-design.md
Normal file
@@ -0,0 +1,371 @@
|
||||
# Magnum Opus v0.2 — Native Swift Email Client
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Build a read-only email client in pure Swift that runs on macOS, iOS, and iPadOS. Prove the on-device architecture: IMAP sync → SQLite cache → SwiftUI three-column UI with threaded message view and full-text search.
|
||||
|
||||
**Architectural pivot from v0.1:** No remote server. All sync, indexing, and storage run on-device. Standard protocols (IMAP) connect directly to mail servers. The v0.1 TypeScript prototype (branch `feature/v0.1-backend-and-macos`) remains as reference.
|
||||
|
||||
**Design document:** `docs/plans/2026-03-10-magnum-opus-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
Everything runs on-device. The app connects directly to IMAP servers via swift-nio-imap, stores messages in SQLite via GRDB, and renders with SwiftUI. No intermediate server, no CLI tool dependencies, no file-based storage layer.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ SwiftUI Apps │
|
||||
│ (macOS / iOS / iPadOS targets) │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌────────────┐ ┌─────────────┐ │
|
||||
│ │ Sidebar │ │ ThreadList │ │ ThreadDetail│ │
|
||||
│ └────┬─────┘ └─────┬──────┘ └──────┬──────┘ │
|
||||
│ └───────────────┴────────────────┘ │
|
||||
│ │ @Observable │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ ViewModels │ │
|
||||
│ └────────┬────────┘ │
|
||||
├───────────────────────┼──────────────────────────┤
|
||||
│ MagnumOpusCore (Swift Package) │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ SyncEngine │ │
|
||||
│ │ (coordinator) │ │
|
||||
│ └───┬─────────┬───┘ │
|
||||
│ ┌────────┘ └────────┐ │
|
||||
│ ┌──────┴──────┐ ┌───────┴───────┐ │
|
||||
│ │ IMAPClient │ │ MailStore │ │
|
||||
│ │ (nio-imap) │ │ (GRDB+FTS5) │ │
|
||||
│ └──────┬──────┘ └───────────────┘ │
|
||||
│ │ │
|
||||
└─────────┼────────────────────────────────────────┘
|
||||
│ TLS
|
||||
▼
|
||||
┌───────────┐
|
||||
│ IMAP │
|
||||
│ Server │
|
||||
└───────────┘
|
||||
```
|
||||
|
||||
**Data flow:** SyncCoordinator triggers IMAPClient → fetches message deltas → writes to MailStore (SQLite) → GRDB ValueObservation pushes updates to @Observable ViewModels → SwiftUI re-renders.
|
||||
|
||||
---
|
||||
|
||||
## Package & Project Structure
|
||||
|
||||
```
|
||||
MagnumOpus/
|
||||
├── Packages/
|
||||
│ └── MagnumOpusCore/
|
||||
│ ├── Package.swift
|
||||
│ ├── Sources/
|
||||
│ │ ├── Models/ ← shared types
|
||||
│ │ ├── IMAPClient/ ← IMAP protocol layer
|
||||
│ │ ├── MailStore/ ← SQLite + GRDB + FTS5
|
||||
│ │ └── SyncEngine/ ← sync coordinator
|
||||
│ └── Tests/
|
||||
│ ├── IMAPClientTests/
|
||||
│ ├── MailStoreTests/
|
||||
│ └── SyncEngineTests/
|
||||
├── Apps/
|
||||
│ ├── macOS/ ← macOS SwiftUI app
|
||||
│ └── iOS/ ← iOS/iPadOS SwiftUI app
|
||||
├── docs/
|
||||
├── Ideas/
|
||||
└── scripts/
|
||||
```
|
||||
|
||||
**Dependency direction:** `SyncEngine` → `IMAPClient` + `MailStore` → `Models`. App targets import `SyncEngine` and `MailStore`.
|
||||
|
||||
---
|
||||
|
||||
## Data Model
|
||||
|
||||
### Core Types (`Models` module)
|
||||
|
||||
- **`Account`** — IMAP/SMTP server config. Credentials stored in Keychain, not SQLite.
|
||||
- **`Mailbox`** — IMAP folder (Inbox, Sent, Drafts, Archive, Trash, custom). Tracks `uidValidity` and `uidNext` for delta sync.
|
||||
- **`Message`** — email with headers, body, attachment metadata. Identified by RFC 5322 `Message-ID`.
|
||||
- **`Thread`** — group of messages linked by `In-Reply-To` / `References` headers.
|
||||
|
||||
### SQLite Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE account (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
email TEXT NOT NULL,
|
||||
imapHost TEXT NOT NULL,
|
||||
imapPort INTEGER NOT NULL,
|
||||
smtpHost TEXT NOT NULL,
|
||||
smtpPort INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE mailbox (
|
||||
id TEXT PRIMARY KEY,
|
||||
accountId TEXT NOT NULL REFERENCES account(id),
|
||||
name TEXT NOT NULL,
|
||||
uidValidity INTEGER NOT NULL,
|
||||
uidNext INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE message (
|
||||
id TEXT PRIMARY KEY,
|
||||
accountId TEXT NOT NULL REFERENCES account(id),
|
||||
mailboxId TEXT NOT NULL REFERENCES mailbox(id),
|
||||
uid INTEGER NOT NULL,
|
||||
messageId TEXT,
|
||||
inReplyTo TEXT,
|
||||
refs TEXT,
|
||||
subject TEXT,
|
||||
fromAddress TEXT,
|
||||
fromName TEXT,
|
||||
toAddresses TEXT,
|
||||
ccAddresses TEXT,
|
||||
date TEXT NOT NULL,
|
||||
snippet TEXT,
|
||||
bodyText TEXT,
|
||||
bodyHtml TEXT,
|
||||
isRead INTEGER NOT NULL DEFAULT 0,
|
||||
isFlagged INTEGER NOT NULL DEFAULT 0,
|
||||
size INTEGER NOT NULL DEFAULT 0,
|
||||
UNIQUE(mailboxId, uid)
|
||||
);
|
||||
|
||||
CREATE TABLE thread (
|
||||
id TEXT PRIMARY KEY,
|
||||
accountId TEXT NOT NULL REFERENCES account(id),
|
||||
subject TEXT,
|
||||
lastDate TEXT NOT NULL,
|
||||
messageCount INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE TABLE threadMessage (
|
||||
threadId TEXT NOT NULL REFERENCES thread(id),
|
||||
messageId TEXT NOT NULL REFERENCES message(id),
|
||||
PRIMARY KEY (threadId, messageId)
|
||||
);
|
||||
|
||||
CREATE VIRTUAL TABLE messageFts USING fts5(
|
||||
subject, fromName, fromAddress, bodyText,
|
||||
content='message', content_rowid='rowid'
|
||||
);
|
||||
```
|
||||
|
||||
### Thread Reconstruction
|
||||
|
||||
Simplified JWZ algorithm — no subject-based fallback grouping (produces false matches):
|
||||
|
||||
1. New message arrives: extract `Message-ID`, `In-Reply-To`, `References`
|
||||
2. Look up existing threads where any member shares a reference
|
||||
3. If found → add to that thread (merge if multiple threads match)
|
||||
4. If not found → create new thread
|
||||
5. Update thread metadata (last date, message count, subject from root message)
|
||||
|
||||
---
|
||||
|
||||
## IMAP Client Layer
|
||||
|
||||
Wraps swift-nio-imap into a clean async/await API:
|
||||
|
||||
```swift
|
||||
actor IMAPClient {
|
||||
init(host: String, port: Int, credentials: Credentials)
|
||||
|
||||
func connect() async throws
|
||||
func disconnect() async throws
|
||||
|
||||
func listMailboxes() async throws -> [MailboxInfo]
|
||||
func selectMailbox(_ name: String) async throws -> MailboxStatus
|
||||
|
||||
func fetchMessages(uids: UIDRange, fields: FetchFields) async throws -> [FetchedMessage]
|
||||
func fetchNew(since uid: UID) async throws -> [FetchedMessage]
|
||||
func fetchFlags(uids: UIDRange) async throws -> [UIDFlagsPair]
|
||||
}
|
||||
```
|
||||
|
||||
### Sync Strategy (Poll-Based for v0.2)
|
||||
|
||||
1. Connect to IMAP server
|
||||
2. `SELECT` each mailbox, compare `UIDVALIDITY` with stored value
|
||||
- Changed → full re-sync of that mailbox (rare — means server rebuilt UIDs)
|
||||
- Same → fetch messages with UID > last known `uidNext`
|
||||
3. Fetch envelopes + snippets eagerly; full bodies lazily
|
||||
4. Sync flag changes (read/unread/starred) bidirectionally
|
||||
5. Disconnect, schedule next poll
|
||||
|
||||
### Eager vs. Lazy Fetching
|
||||
|
||||
- **Eager (on sync):** envelope (from, to, subject, date, message-id, references), flags, snippet (~200 chars of body)
|
||||
- **Lazy (on open):** full body text, HTML body, attachments — fetched when user opens message, then cached in SQLite
|
||||
- **Background fill:** after initial sync, progressively fetch bodies for recent messages (last 30 days) so they're available offline
|
||||
|
||||
### Attachment Caching
|
||||
|
||||
Attachment metadata stored eagerly in SQLite. Binary content downloaded on demand, cached to disk in the app sandbox (not SQLite — attachments can be large).
|
||||
|
||||
---
|
||||
|
||||
## Sync Engine
|
||||
|
||||
Orchestrates the full sync lifecycle:
|
||||
|
||||
```swift
|
||||
@Observable
|
||||
final class SyncCoordinator {
|
||||
private let account: Account
|
||||
private let imapClient: IMAPClient
|
||||
private let store: MailStore
|
||||
|
||||
var syncState: SyncState // .idle, .syncing(progress), .error(Error)
|
||||
var events: AsyncStream<SyncEvent>
|
||||
|
||||
func startPeriodicSync(interval: Duration = .seconds(300))
|
||||
func syncNow() async throws
|
||||
func stopSync()
|
||||
}
|
||||
|
||||
enum SyncEvent {
|
||||
case newMessages(count: Int, mailbox: String)
|
||||
case flagsChanged(messageIds: [String])
|
||||
case syncStarted
|
||||
case syncCompleted
|
||||
case syncFailed(Error)
|
||||
}
|
||||
```
|
||||
|
||||
### MailStore — Read/Write Interface
|
||||
|
||||
```swift
|
||||
@Observable
|
||||
final class MailStore {
|
||||
func threads(in mailbox: String, limit: Int, offset: Int) -> [Thread]
|
||||
func messages(in thread: Thread) -> [Message]
|
||||
func message(id: String) -> Message?
|
||||
func search(query: String) -> [Message]
|
||||
|
||||
func insertMessages(_ messages: [FetchedMessage]) throws
|
||||
func updateFlags(uid: UID, mailbox: String, flags: MessageFlags) throws
|
||||
func storeBody(messageId: String, text: String?, html: String?) throws
|
||||
func reconstructThreads(for messages: [Message]) throws
|
||||
|
||||
func observeThreads(in mailbox: String) -> AsyncValueObservation<[Thread]>
|
||||
}
|
||||
```
|
||||
|
||||
GRDB `ValueObservation` provides live-updating queries. When SyncEngine writes new messages, any SwiftUI view observing that mailbox automatically updates. This replaces the SSE pattern from v0.1.
|
||||
|
||||
---
|
||||
|
||||
## SwiftUI App Layer
|
||||
|
||||
### View Hierarchy
|
||||
|
||||
```
|
||||
AppRoot
|
||||
├── AccountSetupView ← first launch: configure IMAP/SMTP
|
||||
└── MainView (NavigationSplitView)
|
||||
├── Sidebar
|
||||
│ └── MailboxListView ← Inbox, Sent, Drafts, Archive, Trash, custom
|
||||
├── Content
|
||||
│ └── ThreadListView ← threads in selected mailbox
|
||||
└── Detail
|
||||
└── ThreadDetailView ← messages in selected thread
|
||||
└── MessageView ← single message (header, body, attachments)
|
||||
```
|
||||
|
||||
### Platform Adaptation
|
||||
|
||||
`NavigationSplitView` handles platform differences automatically:
|
||||
|
||||
- **macOS:** three-column layout always visible, keyboard shortcuts
|
||||
- **iPad landscape:** three columns, collapsible sidebar
|
||||
- **iPad portrait / iPhone:** push navigation (sidebar → thread list → detail)
|
||||
|
||||
### Account Setup (First Launch)
|
||||
|
||||
1. User enters email address
|
||||
2. Auto-discovery via SRV records + well-known configs (Gmail, Outlook, Fastmail, etc.)
|
||||
3. Fall back to manual IMAP/SMTP host + port entry
|
||||
4. Credentials stored in Keychain
|
||||
5. Initial sync begins immediately after setup
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
SyncCoordinator (background sync)
|
||||
↓ writes
|
||||
MailStore (SQLite via GRDB)
|
||||
↓ ValueObservation
|
||||
@Observable ViewModels
|
||||
↓
|
||||
SwiftUI Views
|
||||
```
|
||||
|
||||
No manual refresh needed. Background sync + GRDB observation = automatic UI updates.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Purpose |
|
||||
|---------|---------|
|
||||
| [swift-nio-imap](https://github.com/apple/swift-nio-imap) | IMAP protocol layer |
|
||||
| [swift-nio](https://github.com/apple/swift-nio) | Networking (transitive) |
|
||||
| [swift-nio-ssl](https://github.com/apple/swift-nio-ssl) | TLS for IMAP/SMTP |
|
||||
| [GRDB.swift](https://github.com/groue/GRDB.swift) | SQLite, FTS5, ValueObservation |
|
||||
|
||||
Four dependencies total (two transitive). No bloat.
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **`IMAPClientTests`** — mock IMAP server via swift-nio-imap test utilities. Verify connection, fetch, flag sync, UID tracking, error handling.
|
||||
- **`MailStoreTests`** — in-memory SQLite. Test insert, query, thread reconstruction, FTS5 search, observation. Most testable layer.
|
||||
- **`SyncEngineTests`** — mock IMAPClient + real MailStore (in-memory). Test full sync flow: delta detection, message storage, flag reconciliation, error recovery.
|
||||
- **UI tests** — minimal for v0.2. Verify navigation flow on both platforms.
|
||||
|
||||
Not tested in v0.2: performance at scale, real IMAP server integration.
|
||||
|
||||
---
|
||||
|
||||
## v0.2 Scope
|
||||
|
||||
### In
|
||||
|
||||
- Single account (IMAP only, SMTP config stored but not used yet)
|
||||
- IMAP sync via swift-nio-imap (poll-based, configurable interval)
|
||||
- SQLite cache with GRDB (envelope eager, body lazy + cached)
|
||||
- FTS5 full-text search
|
||||
- Thread reconstruction (Message-ID / In-Reply-To / References)
|
||||
- Three-column SwiftUI layout (macOS + iOS + iPadOS)
|
||||
- Mailbox navigation (Inbox, Sent, Drafts, Archive, Trash, custom folders)
|
||||
- Message reading with HTML/plain text display
|
||||
- Attachment metadata display (download on demand)
|
||||
- Account setup with auto-discovery
|
||||
- Keychain credential storage
|
||||
- Background body prefetch (last 30 days)
|
||||
|
||||
### Out (Deferred)
|
||||
|
||||
- Compose / reply / forward (needs SMTP client — v0.3)
|
||||
- Triage actions: do / defer / delegate / file / discard (v0.3)
|
||||
- Task management / VTODO (v0.4+)
|
||||
- CalDAV / CardDAV sync (v0.4+)
|
||||
- IMAP IDLE for real-time push (optimization, post-v0.2)
|
||||
- Multiple accounts
|
||||
- Keyboard shortcuts beyond system defaults
|
||||
- Notifications
|
||||
|
||||
---
|
||||
|
||||
## Future Evolution
|
||||
|
||||
v0.2 is the foundation. The path forward:
|
||||
|
||||
- **v0.3:** SMTP client (SwiftNIO-based), compose/reply/forward, basic triage (archive, delete, flag)
|
||||
- **v0.4:** VTODO tasks via CalDAV, unified inbox mixing email + tasks, GTD triage workflow
|
||||
- **v0.5:** Contacts via CardDAV, calendar via CalDAV, delegation workflow
|
||||
- **Later:** IMAP IDLE, multiple accounts, keyboard-first triage, append-only history
|
||||
Reference in New Issue
Block a user