add v0.3 design spec: compose, triage, smtp, action queue

This commit is contained in:
2026-03-13 22:12:47 +01:00
parent d5661459e4
commit 5e1c26aa05

View File

@@ -0,0 +1,566 @@
# Magnum Opus v0.3 — Compose, Triage, and SMTP
**Goal:** Turn the read-only v0.2 email client into a fully functional email client. Add SMTP sending (compose, reply, forward), IMAP write-back (flags, move, delete, append), and an offline-safe action queue. Plain text compose only.
**Builds on:** v0.2 native Swift email client (IMAP sync, GRDB/SQLite, SwiftUI three-column UI, threaded messages, FTS5 search).
---
## Architecture Overview
v0.3 adds three capabilities to the existing architecture:
1. **SMTPClient** — new MagnumOpusCore module for sending email
2. **IMAP write operations** — extend existing IMAPClient with flag/move/append commands
3. **ActionQueue** — persistent queue in SyncEngine for offline-safe action dispatch
```
┌─────────────────────────────────────────────────────────────┐
│ SwiftUI Apps │
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ ComposeView│ │TriageBar │ │ MoveToSheet│ │
│ └─────┬──────┘ └─────┬──────┘ └─────┬──────┘ │
│ └────────────────┼───────────────┘ │
│ ┌─────┴──────┐ │
│ │ ViewModels │ │
│ └─────┬──────┘ │
├─────────────────────────┼────────────────────────────────────┤
│ MagnumOpusCore (Swift Package) │
│ ┌─────┴──────┐ │
│ │ SyncEngine │ │
│ │ │ │
│ ┌─────────┼─────────┐ │ │
│ │ ActionQueue │ │ │
│ │ (offline queue) │ │ │
│ └──┬──────────┬─────┘ │ │
│ ┌────────┘ └──────┐ │ │
│ │ SMTPClient │ │IMAPClient│ │ ┌──────────┐ │
│ │ (NEW) │ │(extended)│ │ │ MailStore │ │
│ └─────┬──────┘ └────┬────┘ │ │(GRDB+FTS5)│ │
│ │ │ │ └───────────┘ │
└─────────┼───────────────┼──────┼─────────────────────────────┘
│ TLS │ TLS
▼ ▼
┌──────────┐ ┌──────────┐
│ SMTP │ │ IMAP │
│ Server │ │ Server │
└──────────┘ └──────────┘
```
**Data flow for write operations:**
1. User action (triage or send) → ViewModel
2. ViewModel → ActionQueue.enqueue() → local SQLite update (immediate UI feedback)
3. ActionQueue dispatcher → SMTPClient or IMAPClient (immediate attempt, queue if offline)
4. Next sync cycle: flush pending actions before fetching new messages
---
## Schema Changes
### Migration v2_smtp: Account SMTP fields
```sql
ALTER TABLE account ADD COLUMN smtpHost TEXT;
ALTER TABLE account ADD COLUMN smtpPort INTEGER;
ALTER TABLE account ADD COLUMN smtpSecurity TEXT; -- "ssl" or "starttls"
```
### Migration v2_draft: Local drafts table
```sql
CREATE TABLE draft (
id TEXT PRIMARY KEY,
accountId TEXT NOT NULL REFERENCES account(id),
inReplyTo TEXT,
forwardOf TEXT,
toAddresses TEXT, -- JSON array of {"name", "address"}
ccAddresses TEXT, -- JSON array of {"name", "address"}
subject TEXT,
bodyText TEXT,
createdAt TEXT NOT NULL,
updatedAt TEXT NOT NULL
);
```
### Migration v2_pendingAction: Offline action queue
```sql
CREATE TABLE pendingAction (
id TEXT PRIMARY KEY,
accountId TEXT NOT NULL REFERENCES account(id),
actionType TEXT NOT NULL, -- "setFlags", "move", "delete", "send", "append"
payload TEXT NOT NULL, -- JSON with action-specific data
createdAt TEXT NOT NULL,
retryCount INTEGER NOT NULL DEFAULT 0,
lastError TEXT
);
```
---
## SMTPClient Module
New module in MagnumOpusCore. Built on SwiftNIO + swift-nio-ssl, mirroring the IMAPClient layering pattern.
### Internal Layers
1. **SMTPConnection** (actor) — NIO `ClientBootstrap` with TLS. Handles implicit SSL (port 465) and STARTTLS (port 587). Sends raw SMTP lines, reads status code + text responses.
2. **SMTPResponseHandler** (ChannelInboundHandler) — SMTP responses are line-based: `<3-digit code> <text>`. Multiline responses use `<code>-<text>` continuation. Buffer lines, deliver complete response via continuation.
3. **SMTPCommandRunner** — sends commands sequentially, waits for response. No tags (unlike IMAP) — SMTP is strictly request/response.
### Public API
```swift
public actor SMTPClient: Sendable {
public init(host: String, port: Int, security: SMTPSecurity, credentials: Credentials)
public func send(message: OutgoingMessage) async throws
public func testConnection() async throws
}
public enum SMTPSecurity: String, Sendable, Codable {
case ssl // implicit TLS, port 465
case starttls // upgrade after connect, port 587
}
public struct OutgoingMessage: Sendable {
public var from: EmailAddress
public var to: [EmailAddress]
public var cc: [EmailAddress]
public var subject: String
public var bodyText: String
public var inReplyTo: String? // Message-ID for threading
public var references: String? // accumulated References header
public var messageId: String // generated Message-ID for this message
}
```
### SMTP Command Sequence
```
connect (TLS or plain)
← 220 server greeting
→ EHLO <hostname>
← 250 capabilities (check for STARTTLS, AUTH mechanisms)
[if STARTTLS: → STARTTLS, ← 220, upgrade TLS, → EHLO again]
→ AUTH PLAIN <base64(username + password)>
← 235 authenticated
→ MAIL FROM:<sender>
← 250 OK
→ RCPT TO:<recipient> (repeat for each to + cc)
← 250 OK
→ DATA
← 354 go ahead
→ <RFC 5322 formatted message>
→ .
← 250 OK (message accepted)
→ QUIT
← 221 bye
```
### Message Formatting
The SMTPClient builds the RFC 5322 message internally:
```
From: "Name" <address>
To: "Name" <address>, ...
Cc: "Name" <address>, ...
Subject: <subject>
Date: <RFC 2822 date>
Message-ID: <generated-id@domain>
In-Reply-To: <original-message-id> (if reply)
References: <accumulated-refs> (if reply)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<body text, quoted-printable encoded>
```
### Authentication
Support AUTH PLAIN (most common) and AUTH LOGIN (fallback). Check server's EHLO capabilities for supported mechanisms. If neither is available, throw an error — no CRAM-MD5 or OAuth in v0.3.
### Testing
Mock SMTP server using NIO: accept connections, verify command sequence, capture the formatted message. Tests verify:
- Connection with SSL and STARTTLS
- Authentication (PLAIN and LOGIN)
- Message formatting (headers, encoding, threading headers)
- Error handling (auth failure, rejected recipient, connection drop)
---
## IMAP Write Operations
Extend the existing `IMAPClient` actor and `IMAPClientProtocol`.
### New Protocol Methods
```swift
public protocol IMAPClientProtocol: Sendable {
// existing v0.2 methods...
// v0.3 write operations
func storeFlags(uid: Int, mailbox: String, add: [String], remove: [String]) async throws
func moveMessage(uid: Int, from: String, to: String) async throws
func copyMessage(uid: Int, from: String, to: String) async throws
func expunge(mailbox: String) async throws
func appendMessage(to mailbox: String, message: Data, flags: [String]) async throws
func capabilities() async throws -> Set<String>
}
```
### IMAP MOVE Support
RFC 6851 MOVE command. Check server capabilities at connect time:
- If `MOVE` is advertised: `UID MOVE <uid> <destination>`
- Otherwise: `UID COPY <uid> <destination>` + `UID STORE <uid> +FLAGS (\Deleted)` + `EXPUNGE`
### Special Folder Detection
Detect well-known folders from mailbox LIST attributes (RFC 6154):
| Attribute | Purpose | Name fallbacks |
|-----------|---------|---------------|
| `\Trash` | Delete target | "Trash", "Deleted Messages", "Bin" |
| `\Archive` | Archive target | "Archive", "All Mail" |
| `\Sent` | Sent mail | "Sent", "Sent Messages", "Sent Mail" |
| `\Drafts` | Drafts | "Drafts" |
| `\Junk` | Spam | "Junk", "Spam" |
Store detected folder names in the MailboxRecord for use by triage actions.
### Testing
Extend `MockIMAPClient` with the new protocol methods. Test:
- Flag store commands (add/remove flags)
- MOVE command generation
- COPY+DELETE+EXPUNGE fallback when MOVE not available
- APPEND command with message data and flags
- Capability detection
---
## ActionQueue
Lives in the `SyncEngine` module. The central dispatcher for all write operations.
### Design
```swift
@MainActor
public final class ActionQueue {
private let store: MailStore
private let imapClient: any IMAPClientProtocol
private let smtpClient: SMTPClient?
/// Enqueue an action applies locally first, then attempts remote dispatch
public func enqueue(_ action: PendingAction) throws
/// Flush all pending actions to server (called by SyncCoordinator before fetch)
public func flush() async
/// Number of pending actions (for UI badge/indicator)
public var pendingCount: Int
}
```
### PendingAction Types
```swift
public struct PendingAction: Sendable, Codable {
var id: String
var accountId: String
var actionType: ActionType
var payload: ActionPayload
var createdAt: Date
}
public enum ActionType: String, Sendable, Codable {
case setFlags
case move
case delete
case send
case append
}
public enum ActionPayload: Sendable, Codable {
case setFlags(uid: Int, mailbox: String, add: [String], remove: [String])
case move(uid: Int, from: String, to: String)
case delete(uid: Int, mailbox: String, trashMailbox: String)
case send(message: OutgoingMessage)
case append(mailbox: String, messageData: String, flags: [String])
}
```
### Two-Phase Execution
When the user performs an action:
1. **Local phase (synchronous, immediate):**
- Update SQLite directly (change flags, update mailboxId for moves, etc.)
- GRDB ValueObservation fires → UI updates instantly
- Create `pendingAction` record in SQLite
2. **Remote phase (async):**
- Attempt to dispatch the action via IMAP/SMTP immediately
- On success: delete the `pendingAction` record
- On failure: leave in queue, increment `retryCount`, store error
### Ordering and Concurrency
- Actions are dispatched in creation order (FIFO), sequential per account
- A `send` followed by `append` (save to Sent) must execute in order
- If an action fails, subsequent actions for the same message are held (no flag change after a failed move)
### Failure Handling
| Error Type | Behavior |
|-----------|----------|
| Network timeout / connection refused | Retry on next flush. Exponential backoff: 30s, 1m, 2m, 4m, 8m, 15m cap. |
| Auth failure (401/403) | Stop queue, surface re-auth prompt to user |
| Permanent failure (message doesn't exist) | Mark failed, remove from queue, surface to user |
| Retry cap exceeded (5 attempts) | Mark failed, surface to user |
### SyncCoordinator Integration
The existing `performSync()` flow becomes:
1. Flush action queue (push local changes to server)
2. Connect to IMAP
3. Fetch deltas (existing v0.2 sync)
4. Disconnect
This ensures the server reflects local changes before we pull new state.
---
## Compose Flow
### ComposeViewModel
```swift
@Observable @MainActor
final class ComposeViewModel {
var to: String = ""
var cc: String = ""
var subject: String = ""
var bodyText: String = ""
var mode: ComposeMode = .new
// Draft state
var draftId: String?
var isDirty: Bool { /* compare current to saved */ }
func saveDraft() throws
func deleteDraft() throws
func send() async throws
}
enum ComposeMode: Sendable {
case new
case reply(to: MessageSummary)
case replyAll(to: MessageSummary)
case forward(of: MessageSummary)
case draft(DraftRecord)
}
```
### Reply Behavior
- **To:** Original sender (reply) or all recipients minus self (reply-all)
- **Subject:** `Re: <original>` (strip duplicate Re:/Fwd: prefixes)
- **Body:** `\n\nOn <date>, <sender> wrote:\n> <line1>\n> <line2>\n...`
- **Headers:** Set `In-Reply-To: <original-message-id>`, append to `References`
### Forward Behavior
- **To:** Empty (user fills in)
- **Subject:** `Fwd: <original>`
- **Body:**
```
\n\n---------- Forwarded message ----------
From: <original sender>
Date: <original date>
Subject: <original subject>
To: <original recipients>
<original body>
```
- **Headers:** No `In-Reply-To`. Set `References` to original Message-ID.
### Send Sequence
Two actions enqueued atomically:
1. `send` — SMTPClient delivers the message
2. `append` — IMAPClient appends the formatted message to the Sent folder with `\Seen` flag
If send succeeds but append fails, the append stays in the queue for retry. The message is delivered — the Sent folder copy is a convenience, not a requirement.
### Draft Auto-Save
- Auto-save to SQLite every 10 seconds while `isDirty`
- On explicit discard: delete draft record, dismiss compose
- On successful send enqueue: delete draft record, dismiss compose
- On close without send: save draft, dismiss compose (draft persists)
### ComposeView
- **macOS:** New window via `openWindow`
- **iOS:** Sheet presentation
- Fields: To, CC (expandable), Subject, body TextEditor
- Toolbar: Send button, Discard button
- Compose is modal — one compose at a time in v0.3
---
## Triage Actions and UI
### Actions
| Action | IMAP Operation | Local Effect | macOS Shortcut |
|--------|---------------|-------------|----------------|
| Archive | MOVE to Archive folder | Update mailboxId | `e` |
| Delete | MOVE to Trash | Update mailboxId | `` |
| Flag/unflag | STORE +/-\Flagged | Toggle isFlagged | `s` |
| Read/unread | STORE +/-\Seen | Toggle isRead | `⇧⌘U` |
| Move to folder | MOVE to chosen folder | Update mailboxId | `⌘⇧M` |
All actions go through the ActionQueue for offline safety.
### UI Integration
**ThreadListView toolbar:**
- Archive, Delete, Flag buttons (visible when thread selected)
- Reply, Reply All, Forward buttons
**Swipe actions on thread rows:**
- Leading swipe: Archive (green)
- Trailing swipe: Delete (red)
**Keyboard shortcuts (macOS):**
- Applied via `.keyboardShortcut()` on toolbar buttons
- Only active when the mail view has focus
**Move-to-folder picker:**
- Sheet with searchable mailbox list
- Triggered by toolbar button or `⌘⇧M`
**Auto-advance:**
- After archiving/deleting, automatically select the next thread in the list
- If no next thread, select previous. If list empty, show empty state.
### Thread-Level vs Message-Level
Triage actions operate on the **entire thread** — all messages in the thread are moved/flagged together. This matches the standard email client behavior for threaded views.
---
## Account Setup Updates
### AccountConfig Model
```swift
public struct AccountConfig: Sendable, Codable, Equatable {
public var id: String
public var name: String
public var email: String
public var imapHost: String
public var imapPort: Int
public var smtpHost: String?
public var smtpPort: Int?
public var smtpSecurity: SMTPSecurity?
}
```
SMTP fields are optional for backwards compatibility with v0.2 accounts.
### Auto-Discovery Extension
The existing `AutoDiscovery.parseISPDBXML` already receives the full XML. Extend it to also extract the `<outgoingServer type="smtp">` block:
```swift
struct DiscoveredConfig: Sendable {
var imap: DiscoveredServer?
var smtp: DiscoveredServer?
}
```
### AccountSetupView Updates
- Show SMTP fields alongside IMAP in manual mode
- Auto-discovery fills both IMAP and SMTP
- Test both connections before saving
- Existing v0.2 accounts: prompt to add SMTP settings when user first opens compose
---
## Testing Strategy
### New Test Targets
- **SMTPClientTests** — mock SMTP server, test connection/auth/send/formatting/errors
- Extended **IMAPClientTests** — test flag store, move, copy, append, capability detection
- Extended **SyncEngineTests** — test ActionQueue enqueue/flush/retry/failure handling
- Extended **App tests** — ComposeViewModel: reply/forward prefill, draft save/load, send validation
### Key Test Scenarios
1. Send plain text email → verify RFC 5322 formatting, SMTP command sequence
2. Reply to message → verify threading headers (In-Reply-To, References)
3. Archive while offline → local update immediate, action queued, flushed on reconnect
4. Send fails (auth error) → queue stops, user prompted, retry after re-auth
5. Move message → MOVE command if supported, COPY+DELETE fallback if not
6. Draft auto-save → save on timer, load on resume, delete on send
---
## Dependencies
No new external dependencies. v0.3 uses the same four packages as v0.2:
| Package | Purpose |
|---------|---------|
| swift-nio-imap | IMAP protocol (existing) |
| swift-nio | Networking (transitive, also used for SMTP) |
| swift-nio-ssl | TLS for IMAP and SMTP (existing) |
| GRDB.swift | SQLite, FTS5, ValueObservation (existing) |
The SMTPClient is built on swift-nio and swift-nio-ssl directly — no SMTP-specific dependency needed.
---
## v0.3 Scope
### In
- SMTPClient module (NIO-based, SSL/STARTTLS, AUTH PLAIN/LOGIN)
- Compose: new message, reply, reply-all, forward (plain text)
- Local drafts with auto-save (SQLite, not synced to IMAP)
- IMAP APPEND to Sent folder after send
- IMAP write-back: flag changes, move, delete
- IMAP MOVE with COPY+DELETE fallback
- ActionQueue: local-first, immediate remote dispatch, offline queue with retry
- Special folder detection (Trash, Archive, Sent via LIST attributes)
- Triage: archive, delete, flag, mark read/unread, move to folder
- Keyboard shortcuts for triage (macOS)
- Swipe actions on thread rows
- Auto-advance after triage
- Account setup extended for SMTP (auto-discovery + manual)
- Schema migration for SMTP fields, drafts, pending actions
### Out (Deferred)
- Rich text / Markdown compose (v0.3+)
- IMAP draft sync (local only in v0.3)
- Attachment sending (v0.3+)
- OAuth / XOAUTH2 authentication
- IMAP IDLE push
- Multiple accounts
- GTD triage (defer/delegate — v0.4)
- VTODO / CalDAV (v0.4)