From b866c3bb8ea64f6735218d7b14ee267db9691414 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Felix=20F=C3=B6rtsch?= Date: Sat, 14 Mar 2026 11:54:26 +0100 Subject: [PATCH] add v0.5 implementation plan: attachments, MIME parsing, IMAP IDLE --- .../2026-03-14-v0.5-implementation-plan.md | 2902 +++++++++++++++++ 1 file changed, 2902 insertions(+) create mode 100644 docs/plans/2026-03-14-v0.5-implementation-plan.md diff --git a/docs/plans/2026-03-14-v0.5-implementation-plan.md b/docs/plans/2026-03-14-v0.5-implementation-plan.md new file mode 100644 index 0000000..66d3c9a --- /dev/null +++ b/docs/plans/2026-03-14-v0.5-implementation-plan.md @@ -0,0 +1,2902 @@ +# v0.5 Attachments & IMAP IDLE — Implementation Plan + +> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE. + +**Architecture:** Three subsystems: (1) MIMEParser module for parsing multipart MIME messages, (2) attachment sending via multipart/mixed MessageFormatter extension, (3) IMAPIdleClient actor for persistent INBOX monitoring. MIMEParser is a new module with no external dependencies. IDLE uses a separate NIO connection from the sync connection. + +**Tech Stack:** Swift 6, swift-nio, swift-nio-imap, swift-nio-ssl, GRDB.swift, Swift Testing + +--- + +## File Structure + +### New Files +| File | Responsibility | +|------|---------------| +| `Sources/MIMEParser/MIMETypes.swift` | Public types: MIMEMessage, MIMEPart, MIMEAttachment, TransferEncoding, ContentDisposition | +| `Sources/MIMEParser/MIMEParser.swift` | Static parse/decode/boundary functions | +| `Sources/MIMEParser/RFC2047Decoder.swift` | Decode `=?charset?encoding?text?=` encoded words in filenames | +| `Sources/IMAPClient/IMAPIdleClient.swift` | Actor managing dedicated IDLE connection | +| `Sources/IMAPClient/IMAPIdleHandler.swift` | NIO ChannelInboundHandler for IDLE responses, delivers AsyncStream | +| `Tests/MIMEParserTests/MIMEParserTests.swift` | MIME parsing tests | +| `Tests/MIMEParserTests/RFC2047DecoderTests.swift` | RFC 2047 filename decoding tests | +| `Tests/IMAPClientTests/IMAPIdleClientTests.swift` | IDLE client tests (using mock handler) | + +### Modified Files +| File | Changes | +|------|---------| +| `Package.swift` | Add MIMEParser target + test target | +| `Sources/MailStore/DatabaseSetup.swift` | Add v4_attachment migration | +| `Sources/MailStore/Records/AttachmentRecord.swift` | Add sectionPath column | +| `Sources/MailStore/Records/MessageRecord.swift` | Add hasAttachments column | +| `Sources/MailStore/MailStore.swift` | Add attachment CRUD, storeBodyWithAttachments, attachment download query | +| `Sources/MailStore/Queries.swift` | Read hasAttachments in toMessageSummary | +| `Sources/IMAPClient/IMAPClientProtocol.swift` | Add fetchFullMessage, fetchSection | +| `Sources/IMAPClient/IMAPClient.swift` | Implement fetchFullMessage, fetchSection; remove body from fetchEnvelopes | +| `Sources/SMTPClient/MessageFormatter.swift` | Add formatMultipart, base64Encode, 25MB guard | +| `Sources/SyncEngine/SyncCoordinator.swift` | MIME-aware prefetchBodies, IDLE start/stop, attachment download, credentials | +| `Tests/SyncEngineTests/MockIMAPClient.swift` | Add fetchFullMessage, fetchSection mocks | +| `Tests/SMTPClientTests/MessageFormatterTests.swift` | Add multipart formatting + size guard tests | +| `Tests/SyncEngineTests/SyncCoordinatorTests.swift` | Add MIME sync + IDLE integration tests | +| `Tests/MailStoreTests/MigrationTests.swift` | Add v4_attachment migration test | + +### Deferred to App Layer +The spec's UI changes (ComposeViewModel attachments, attachment strip in ThreadDetail, file picker, paperclip icon in message list, inline image cid: replacement) are app-layer SwiftUI code outside MagnumOpusCore. They depend on the core infrastructure built in this plan. They will be wired up when integrating v0.5 into the app targets — same pattern as v0.3/v0.4 where core logic ships first, then UI wraps it. + +--- + +## Chunk 1: MIMEParser Module — Types & Basic Parsing + +### Task 1: Add MIMEParser target to Package.swift + +**Files:** +- Modify: `Package.swift` + +- [ ] **Step 1: Add MIMEParser target and test target** + +In `Package.swift`, add: +- A product: `.library(name: "MIMEParser", targets: ["MIMEParser"])` +- A target: `.target(name: "MIMEParser")` (no dependencies — pure Swift) +- A test target: `.testTarget(name: "MIMEParserTests", dependencies: ["MIMEParser"])` + +Add after the existing `SMTPClient` product/target entries. + +- [ ] **Step 2: Create source directory** + +```bash +mkdir -p Packages/MagnumOpusCore/Sources/MIMEParser +mkdir -p Packages/MagnumOpusCore/Tests/MIMEParserTests +``` + +- [ ] **Step 3: Verify build** + +```bash +cd Packages/MagnumOpusCore +swift build +``` + +Expected: Build succeeds (empty target is fine with directory existing). + +- [ ] **Step 4: Commit** + +```bash +git add Package.swift Sources/MIMEParser Tests/MIMEParserTests +git commit -m "add MIMEParser module target to Package.swift" +``` + +--- + +### Task 2: MIME types + +**Files:** +- Create: `Sources/MIMEParser/MIMETypes.swift` + +- [ ] **Step 1: Write MIMETypes.swift** + +```swift +import Foundation + +public enum TransferEncoding: String, Sendable { + case base64 + case quotedPrintable = "quoted-printable" + case sevenBit = "7bit" + case eightBit = "8bit" + case binary +} + +public enum ContentDisposition: Sendable { + case inline + case attachment +} + +public struct MIMEPart: Sendable { + public var headers: [String: String] + public var contentType: String + public var charset: String? + public var transferEncoding: TransferEncoding + public var disposition: ContentDisposition? + public var filename: String? + public var contentId: String? + public var body: Data + public var subparts: [MIMEPart] + + public init( + headers: [String: String] = [:], + contentType: String = "text/plain", + charset: String? = "utf-8", + transferEncoding: TransferEncoding = .sevenBit, + disposition: ContentDisposition? = nil, + filename: String? = nil, + contentId: String? = nil, + body: Data = Data(), + subparts: [MIMEPart] = [] + ) { + self.headers = headers + self.contentType = contentType + self.charset = charset + self.transferEncoding = transferEncoding + self.disposition = disposition + self.filename = filename + self.contentId = contentId + self.body = body + self.subparts = subparts + } +} + +public struct MIMEAttachment: Sendable { + public var filename: String + public var mimeType: String + public var size: Int + public var contentId: String? + public var sectionPath: String + public var isInline: Bool + + public init( + filename: String, + mimeType: String, + size: Int, + contentId: String? = nil, + sectionPath: String, + isInline: Bool = false + ) { + self.filename = filename + self.mimeType = mimeType + self.size = size + self.contentId = contentId + self.sectionPath = sectionPath + self.isInline = isInline + } +} + +public struct MIMEMessage: Sendable { + public var headers: [String: String] + public var parts: [MIMEPart] + public var textBody: String? + public var htmlBody: String? + public var attachments: [MIMEAttachment] + public var inlineImages: [MIMEAttachment] + + public init( + headers: [String: String] = [:], + parts: [MIMEPart] = [], + textBody: String? = nil, + htmlBody: String? = nil, + attachments: [MIMEAttachment] = [], + inlineImages: [MIMEAttachment] = [] + ) { + self.headers = headers + self.parts = parts + self.textBody = textBody + self.htmlBody = htmlBody + self.attachments = attachments + self.inlineImages = inlineImages + } +} +``` + +- [ ] **Step 2: Verify build** + +```bash +cd Packages/MagnumOpusCore +swift build +``` + +- [ ] **Step 3: Commit** + +```bash +git add Sources/MIMEParser/MIMETypes.swift +git commit -m "add MIME types: MIMEMessage, MIMEPart, MIMEAttachment, TransferEncoding" +``` + +--- + +### Task 3: RFC 2047 decoder + +**Files:** +- Create: `Sources/MIMEParser/RFC2047Decoder.swift` +- Create: `Tests/MIMEParserTests/RFC2047DecoderTests.swift` + +- [ ] **Step 1: Write failing tests for RFC 2047 decoding** + +```swift +import Testing +import Foundation +@testable import MIMEParser + +@Suite("RFC2047Decoder") +struct RFC2047DecoderTests { + + @Test("plain ASCII filename passes through unchanged") + func plainAscii() { + let result = RFC2047Decoder.decode("report.pdf") + #expect(result == "report.pdf") + } + + @Test("base64 encoded UTF-8 filename decoded correctly") + func base64Utf8() { + // "Bericht.pdf" in base64 + let encoded = "=?utf-8?B?QmVyaWNodC5wZGY=?=" + let result = RFC2047Decoder.decode(encoded) + #expect(result == "Bericht.pdf") + } + + @Test("quoted-printable encoded UTF-8 filename decoded correctly") + func quotedPrintableUtf8() { + // "Grüße.txt" — ü = =C3=BC, ß = =C3=9F + let encoded = "=?utf-8?Q?Gr=C3=BC=C3=9Fe.txt?=" + let result = RFC2047Decoder.decode(encoded) + #expect(result == "Grüße.txt") + } + + @Test("multiple encoded words concatenated") + func multipleEncodedWords() { + let encoded = "=?utf-8?B?SGVsbG8=?= =?utf-8?B?V29ybGQ=?=" + let result = RFC2047Decoder.decode(encoded) + #expect(result == "HelloWorld") + } + + @Test("ISO-8859-1 encoded filename decoded correctly") + func iso88591() { + // "café" — é = 0xE9 in ISO-8859-1, base64 of "café" in ISO-8859-1 is "Y2Fm6Q==" + let encoded = "=?iso-8859-1?B?Y2Fm6Q==?=" + let result = RFC2047Decoder.decode(encoded) + #expect(result == "café") + } + + @Test("underscores in Q-encoding replaced with spaces") + func qEncodingUnderscores() { + let encoded = "=?utf-8?Q?my_file_name.pdf?=" + let result = RFC2047Decoder.decode(encoded) + #expect(result == "my file name.pdf") + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +```bash +cd Packages/MagnumOpusCore +swift test --filter RFC2047DecoderTests 2>&1 | tail -5 +``` + +Expected: Compilation error — `RFC2047Decoder` not defined. + +- [ ] **Step 3: Implement RFC2047Decoder** + +```swift +import Foundation + +public enum RFC2047Decoder { + /// Decode RFC 2047 encoded words in a string. + /// Pattern: =?charset?encoding?encoded_text?= + public static func decode(_ input: String) -> String { + let pattern = #"=\?([^?]+)\?([BbQq])\?([^?]*)\?="# + guard let regex = try? NSRegularExpression(pattern: pattern) else { + return input + } + + let nsInput = input as NSString + let matches = regex.matches(in: input, range: NSRange(location: 0, length: nsInput.length)) + + guard !matches.isEmpty else { return input } + + var result = "" + var lastEnd = 0 + + for match in matches { + let matchRange = match.range + // Add any non-encoded text between matches (skip whitespace between adjacent encoded words) + let gap = nsInput.substring(with: NSRange(location: lastEnd, length: matchRange.location - lastEnd)) + let trimmedGap = gap.trimmingCharacters(in: .whitespaces) + if !trimmedGap.isEmpty || lastEnd == 0 { + // Only add gap if it's not just whitespace between encoded words + if lastEnd == 0 && matchRange.location > 0 { + result += gap + } else if !trimmedGap.isEmpty { + result += gap + } + } + + let charset = nsInput.substring(with: match.range(at: 1)) + let encoding = nsInput.substring(with: match.range(at: 2)).uppercased() + let encodedText = nsInput.substring(with: match.range(at: 3)) + + let cfEncoding = CFStringConvertIANACharSetNameToEncoding(charset as CFString) + let nsEncoding = CFStringConvertEncodingToNSStringEncoding(cfEncoding) + + let decoded: String? + if encoding == "B" { + guard let data = Data(base64Encoded: encodedText) else { + result += nsInput.substring(with: matchRange) + lastEnd = matchRange.location + matchRange.length + continue + } + decoded = String(data: data, encoding: String.Encoding(rawValue: nsEncoding)) + } else { + // Q encoding: like quoted-printable but underscores represent spaces + let withSpaces = encodedText.replacingOccurrences(of: "_", with: " ") + let data = decodeQuotedPrintableBytes(withSpaces) + decoded = String(data: data, encoding: String.Encoding(rawValue: nsEncoding)) + } + + result += decoded ?? nsInput.substring(with: matchRange) + lastEnd = matchRange.location + matchRange.length + } + + // Append any trailing non-encoded text + if lastEnd < nsInput.length { + result += nsInput.substring(from: lastEnd) + } + + return result + } + + private static func decodeQuotedPrintableBytes(_ input: String) -> Data { + var data = Data() + var i = input.startIndex + while i < input.endIndex { + if input[i] == "=" { + let hexStart = input.index(after: i) + guard hexStart < input.endIndex else { + data.append(contentsOf: "=".utf8) + break + } + let hexEnd = input.index(hexStart, offsetBy: 1, limitedBy: input.endIndex) ?? input.endIndex + guard hexEnd < input.endIndex else { + data.append(contentsOf: String(input[i...]).utf8) + break + } + let nextAfterHex = input.index(after: hexEnd) + let hex = String(input[hexStart...hexEnd]) + if let byte = UInt8(hex, radix: 16) { + data.append(byte) + i = nextAfterHex + } else { + data.append(contentsOf: "=".utf8) + i = hexStart + } + } else { + data.append(contentsOf: String(input[i]).utf8) + i = input.index(after: i) + } + } + return data + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +```bash +cd Packages/MagnumOpusCore +swift test --filter RFC2047DecoderTests 2>&1 | tail -5 +``` + +Expected: All 6 tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add Sources/MIMEParser/RFC2047Decoder.swift Tests/MIMEParserTests/RFC2047DecoderTests.swift +git commit -m "add RFC 2047 encoded word decoder for MIME filenames" +``` + +--- + +### Task 4: MIMEParser core — header parsing, content decoding, boundary generation + +**Files:** +- Create: `Sources/MIMEParser/MIMEParser.swift` +- Create: `Tests/MIMEParserTests/MIMEParserTests.swift` + +- [ ] **Step 1: Write failing tests for content decoding and boundary generation** + +```swift +import Testing +import Foundation +@testable import MIMEParser + +@Suite("MIMEParser") +struct MIMEParserTests { + + // MARK: - Content Decoding + + @Test("decode base64 content") + func decodeBase64() { + let encoded = "SGVsbG8gV29ybGQ=" + let data = MIMEParser.decodeContent(encoded, encoding: .base64) + #expect(String(data: data, encoding: .utf8) == "Hello World") + } + + @Test("decode quoted-printable content") + func decodeQuotedPrintable() { + let encoded = "Gr=C3=BC=C3=9Fe" + let data = MIMEParser.decodeContent(encoded, encoding: .quotedPrintable) + #expect(String(data: data, encoding: .utf8) == "Grüße") + } + + @Test("decode 7bit content passes through") + func decode7bit() { + let text = "Hello World" + let data = MIMEParser.decodeContent(text, encoding: .sevenBit) + #expect(String(data: data, encoding: .utf8) == "Hello World") + } + + @Test("boundary generation produces unique strings with =_ prefix") + func boundaryGeneration() { + let b1 = MIMEParser.generateBoundary() + let b2 = MIMEParser.generateBoundary() + #expect(b1 != b2) + #expect(b1.hasPrefix("=_MagnumOpus_")) + #expect(b2.hasPrefix("=_MagnumOpus_")) + } + + // MARK: - Single-part Parsing + + @Test("parse single-part text/plain message") + func parseSinglePartText() { + let raw = """ + Content-Type: text/plain; charset=utf-8\r + Content-Transfer-Encoding: 7bit\r + \r + Hello, this is the body. + """ + let message = MIMEParser.parse(raw) + #expect(message.textBody == "Hello, this is the body.") + #expect(message.htmlBody == nil) + #expect(message.attachments.isEmpty) + } + + // MARK: - Multipart Parsing + + @Test("parse multipart/mixed with text and one attachment") + func parseMultipartMixed() { + let raw = """ + Content-Type: multipart/mixed; boundary="----boundary123"\r + \r + ------boundary123\r + Content-Type: text/plain; charset=utf-8\r + Content-Transfer-Encoding: 7bit\r + \r + Hello from the body.\r + ------boundary123\r + Content-Type: application/pdf; name="report.pdf"\r + Content-Disposition: attachment; filename="report.pdf"\r + Content-Transfer-Encoding: base64\r + \r + SGVsbG8=\r + ------boundary123-- + """ + let message = MIMEParser.parse(raw) + #expect(message.textBody == "Hello from the body.") + #expect(message.attachments.count == 1) + #expect(message.attachments.first?.filename == "report.pdf") + #expect(message.attachments.first?.mimeType == "application/pdf") + #expect(message.attachments.first?.sectionPath == "2") + #expect(message.attachments.first?.isInline == false) + } + + @Test("parse multipart/alternative extracts text and html bodies") + func parseMultipartAlternative() { + let raw = """ + Content-Type: multipart/alternative; boundary="alt-boundary"\r + \r + --alt-boundary\r + Content-Type: text/plain; charset=utf-8\r + \r + Plain text body\r + --alt-boundary\r + Content-Type: text/html; charset=utf-8\r + \r +

HTML body

\r + --alt-boundary-- + """ + let message = MIMEParser.parse(raw) + #expect(message.textBody == "Plain text body") + #expect(message.htmlBody == "

HTML body

") + #expect(message.attachments.isEmpty) + } + + @Test("parse multipart/related with inline image") + func parseMultipartRelated() { + let raw = """ + Content-Type: multipart/related; boundary="rel-boundary"\r + \r + --rel-boundary\r + Content-Type: text/html; charset=utf-8\r + \r +

Image:

\r + --rel-boundary\r + Content-Type: image/png\r + Content-ID: \r + Content-Disposition: inline\r + Content-Transfer-Encoding: base64\r + \r + iVBORw0KGgo=\r + --rel-boundary-- + """ + let message = MIMEParser.parse(raw) + #expect(message.htmlBody == "

Image:

") + #expect(message.inlineImages.count == 1) + #expect(message.inlineImages.first?.contentId == "img001") + #expect(message.inlineImages.first?.isInline == true) + } + + @Test("parse nested multipart/mixed containing multipart/alternative") + func parseNestedMultipart() { + let raw = """ + Content-Type: multipart/mixed; boundary="outer"\r + \r + --outer\r + Content-Type: multipart/alternative; boundary="inner"\r + \r + --inner\r + Content-Type: text/plain\r + \r + Plain text\r + --inner\r + Content-Type: text/html\r + \r +

HTML

\r + --inner--\r + --outer\r + Content-Type: application/pdf; name="doc.pdf"\r + Content-Disposition: attachment; filename="doc.pdf"\r + Content-Transfer-Encoding: base64\r + \r + AAAA\r + --outer-- + """ + let message = MIMEParser.parse(raw) + #expect(message.textBody == "Plain text") + #expect(message.htmlBody == "

HTML

") + #expect(message.attachments.count == 1) + #expect(message.attachments.first?.filename == "doc.pdf") + } + + @Test("section paths assigned correctly for nested parts") + func sectionPaths() { + let raw = """ + Content-Type: multipart/mixed; boundary="outer"\r + \r + --outer\r + Content-Type: text/plain\r + \r + Body text\r + --outer\r + Content-Type: application/pdf; name="a.pdf"\r + Content-Disposition: attachment; filename="a.pdf"\r + Content-Transfer-Encoding: base64\r + \r + AAAA\r + --outer\r + Content-Type: image/jpeg; name="b.jpg"\r + Content-Disposition: attachment; filename="b.jpg"\r + Content-Transfer-Encoding: base64\r + \r + BBBB\r + --outer-- + """ + let message = MIMEParser.parse(raw) + #expect(message.attachments.count == 2) + #expect(message.attachments[0].sectionPath == "2") + #expect(message.attachments[1].sectionPath == "3") + } + + @Test("extract filename from Content-Type name parameter when no Content-Disposition") + func filenameFromContentType() { + let raw = """ + Content-Type: multipart/mixed; boundary="bound"\r + \r + --bound\r + Content-Type: text/plain\r + \r + Body\r + --bound\r + Content-Type: application/octet-stream; name="data.bin"\r + Content-Transfer-Encoding: base64\r + \r + AAAA\r + --bound-- + """ + let message = MIMEParser.parse(raw) + #expect(message.attachments.count == 1) + #expect(message.attachments.first?.filename == "data.bin") + } + + @Test("estimate decoded size from base64 content") + func base64SizeEstimate() { + // 8 base64 chars = 6 decoded bytes + let raw = """ + Content-Type: multipart/mixed; boundary="bound"\r + \r + --bound\r + Content-Type: text/plain\r + \r + Body\r + --bound\r + Content-Type: application/pdf; name="f.pdf"\r + Content-Disposition: attachment; filename="f.pdf"\r + Content-Transfer-Encoding: base64\r + \r + AAAAAAAA\r + --bound-- + """ + let message = MIMEParser.parse(raw) + #expect(message.attachments.first?.size == 6) + } + + @Test("handle malformed MIME gracefully — missing boundary") + func malformedMissingBoundary() { + let raw = """ + Content-Type: multipart/mixed\r + \r + Some text without proper boundary markers. + """ + let message = MIMEParser.parse(raw) + // Should not crash; treat as single-part + #expect(message.attachments.isEmpty) + } + + @Test("RFC 2047 encoded filename decoded") + func rfc2047Filename() { + let raw = """ + Content-Type: multipart/mixed; boundary="bound"\r + \r + --bound\r + Content-Type: text/plain\r + \r + Body\r + --bound\r + Content-Type: application/pdf; name="=?utf-8?B?QmVyaWNodC5wZGY=?="\r + Content-Disposition: attachment; filename="=?utf-8?B?QmVyaWNodC5wZGY=?="\r + Content-Transfer-Encoding: base64\r + \r + AAAA\r + --bound-- + """ + let message = MIMEParser.parse(raw) + #expect(message.attachments.first?.filename == "Bericht.pdf") + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MIMEParserTests 2>&1 | tail -5 +``` + +Expected: Compilation error — `MIMEParser` not defined. + +- [ ] **Step 3: Implement MIMEParser** + +```swift +import Foundation + +public enum MIMEParser { + + // MARK: - Public API + + /// Parse a raw MIME message into a structured tree of parts + public static func parse(_ rawMessage: String) -> MIMEMessage { + let (headers, body) = splitHeadersAndBody(rawMessage) + let contentType = headers["content-type"] ?? "text/plain" + + if contentType.lowercased().contains("multipart/") { + guard let boundary = extractBoundary(contentType) else { + // Malformed: multipart without boundary — treat as plain text + return MIMEMessage( + headers: headers, + textBody: body.trimmingCharacters(in: .whitespacesAndNewlines) + ) + } + let parts = splitOnBoundary(body, boundary: boundary) + let parsedParts = parts.enumerated().map { (index, partString) in + parsePart(partString, sectionPrefix: "", index: index + 1) + } + + var message = MIMEMessage(headers: headers, parts: parsedParts) + extractBodiesAndAttachments(from: parsedParts, contentType: contentType, into: &message, sectionPrefix: "") + return message + } else { + // Single-part message + let transferEncoding = parseTransferEncoding(headers["content-transfer-encoding"]) + let decoded = decodeContent(body, encoding: transferEncoding) + + if contentType.lowercased().contains("text/html") { + return MIMEMessage(headers: headers, htmlBody: String(data: decoded, encoding: .utf8)) + } else { + return MIMEMessage( + headers: headers, + textBody: String(data: decoded, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + ) + } + } + } + + /// Decode content based on Content-Transfer-Encoding + public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data { + switch encoding { + case .base64: + let cleaned = content.filter { !$0.isWhitespace } + return Data(base64Encoded: cleaned) ?? Data(content.utf8) + case .quotedPrintable: + return decodeQuotedPrintable(content) + case .sevenBit, .eightBit, .binary: + return Data(content.utf8) + } + } + + /// Generate a unique MIME boundary string + public static func generateBoundary() -> String { + "=_MagnumOpus_\(UUID().uuidString)" + } + + // MARK: - Header Parsing + + private static func splitHeadersAndBody(_ raw: String) -> ([String: String], String) { + // Split on first blank line (CRLF CRLF or LF LF) + let separator: String + if raw.contains("\r\n\r\n") { + separator = "\r\n\r\n" + } else if raw.contains("\n\n") { + separator = "\n\n" + } else { + return ([:], raw) + } + + guard let range = raw.range(of: separator) else { + return ([:], raw) + } + + let headerSection = String(raw[raw.startIndex.. [String: String] { + var headers: [String: String] = [:] + let lineBreak = section.contains("\r\n") ? "\r\n" : "\n" + let lines = section.components(separatedBy: lineBreak) + + var currentKey: String? + var currentValue: String = "" + + for line in lines { + if line.isEmpty { continue } + + if line.first == " " || line.first == "\t" { + // Continuation of previous header (folded) + currentValue += " " + line.trimmingCharacters(in: .whitespaces) + } else if let colonIndex = line.firstIndex(of: ":") { + // Save previous header + if let key = currentKey { + headers[key.lowercased()] = currentValue + } + currentKey = String(line[.. String? { + // Look for boundary="value" or boundary=value + let lower = contentType.lowercased() + guard let boundaryRange = lower.range(of: "boundary=") else { return nil } + var value = String(contentType[boundaryRange.upperBound...]) + + // Strip leading quote + if value.hasPrefix("\"") { + value = String(value.dropFirst()) + if let endQuote = value.firstIndex(of: "\"") { + value = String(value[.. [String] { + let delimiter = "--\(boundary)" + let terminator = "--\(boundary)--" + let lineBreak = body.contains("\r\n") ? "\r\n" : "\n" + + var parts: [String] = [] + let lines = body.components(separatedBy: lineBreak) + var currentPart: [String]? = nil + + for line in lines { + let trimmed = line.trimmingCharacters(in: .whitespaces) + if trimmed == terminator || trimmed.hasPrefix(terminator) { + if let part = currentPart { + parts.append(part.joined(separator: lineBreak)) + } + break + } else if trimmed == delimiter || trimmed.hasPrefix(delimiter) { + if let part = currentPart { + parts.append(part.joined(separator: lineBreak)) + } + currentPart = [] + } else if currentPart != nil { + currentPart!.append(line) + } + } + + return parts + } + + // MARK: - Part Parsing + + private static func parsePart(_ partString: String, sectionPrefix: String, index: Int) -> MIMEPart { + let (headers, body) = splitHeadersAndBody(partString) + let contentType = headers["content-type"] ?? "text/plain" + let transferEncoding = parseTransferEncoding(headers["content-transfer-encoding"]) + let charset = extractParameter(contentType, name: "charset") + let disposition = parseDisposition(headers["content-disposition"]) + let contentId = extractContentId(headers["content-id"]) + + var filename = extractParameter(headers["content-disposition"] ?? "", name: "filename") + if filename == nil { + filename = extractParameter(contentType, name: "name") + } + // Decode RFC 2047 encoded filenames + if let encoded = filename { + filename = RFC2047Decoder.decode(encoded) + } + + let section = sectionPrefix.isEmpty ? "\(index)" : "\(sectionPrefix).\(index)" + + // Check for nested multipart + if contentType.lowercased().contains("multipart/") { + if let boundary = extractBoundary(contentType) { + let subparts = splitOnBoundary(body, boundary: boundary) + let parsedSubparts = subparts.enumerated().map { (i, s) in + parsePart(s, sectionPrefix: section, index: i + 1) + } + return MIMEPart( + headers: headers, + contentType: contentType.components(separatedBy: ";").first?.trimmingCharacters(in: .whitespaces).lowercased() ?? contentType, + charset: charset, + transferEncoding: transferEncoding, + disposition: disposition, + filename: filename, + contentId: contentId, + body: Data(), + subparts: parsedSubparts + ) + } + } + + let decodedBody = decodeContent(body, encoding: transferEncoding) + let baseContentType = contentType.components(separatedBy: ";").first?.trimmingCharacters(in: .whitespaces).lowercased() ?? contentType + + return MIMEPart( + headers: headers, + contentType: baseContentType, + charset: charset, + transferEncoding: transferEncoding, + disposition: disposition, + filename: filename, + contentId: contentId, + body: decodedBody, + subparts: [] + ) + } + + // MARK: - Body & Attachment Extraction + + private static func extractBodiesAndAttachments( + from parts: [MIMEPart], + contentType: String, + into message: inout MIMEMessage, + sectionPrefix: String + ) { + let lowerType = contentType.lowercased() + + if lowerType.contains("multipart/alternative") { + for part in parts { + if !part.subparts.isEmpty { + extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "") + } else if part.contentType == "text/plain" && message.textBody == nil { + message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } else if part.contentType == "text/html" && message.htmlBody == nil { + message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } + } + } else if lowerType.contains("multipart/related") { + // First part is the HTML body, rest are inline resources + for (index, part) in parts.enumerated() { + if index == 0 { + if !part.subparts.isEmpty { + extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "") + } else if part.contentType == "text/html" { + message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } else if part.contentType == "text/plain" { + message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } + } else { + let sectionIndex = sectionPrefix.isEmpty ? "\(index + 1)" : "\(sectionPrefix).\(index + 1)" + let attachment = MIMEAttachment( + filename: part.filename ?? "inline-\(index)", + mimeType: part.contentType, + size: estimateDecodedSize(part), + contentId: part.contentId, + sectionPath: sectionIndex, + isInline: true + ) + message.inlineImages.append(attachment) + } + } + } else { + // multipart/mixed or unknown multipart + var bodyFound = false + for (index, part) in parts.enumerated() { + let sectionIndex = sectionPrefix.isEmpty ? "\(index + 1)" : "\(sectionPrefix).\(index + 1)" + + if !part.subparts.isEmpty { + // Nested multipart — recurse + extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "") + bodyFound = true + } else if !bodyFound && part.disposition != .attachment && part.contentType.hasPrefix("text/") { + if part.contentType == "text/html" { + message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } else { + message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines) + } + bodyFound = true + } else if part.disposition == .attachment || part.filename != nil || !part.contentType.hasPrefix("text/") { + let attachment = MIMEAttachment( + filename: part.filename ?? "attachment-\(index + 1)", + mimeType: part.contentType, + size: estimateDecodedSize(part), + contentId: part.contentId, + sectionPath: sectionIndex, + isInline: part.disposition == .inline + ) + if part.disposition == .inline { + message.inlineImages.append(attachment) + } else { + message.attachments.append(attachment) + } + } + } + } + } + + // MARK: - Helper Functions + + private static func parseTransferEncoding(_ value: String?) -> TransferEncoding { + guard let value = value?.trimmingCharacters(in: .whitespaces).lowercased() else { return .sevenBit } + return TransferEncoding(rawValue: value) ?? .sevenBit + } + + private static func parseDisposition(_ value: String?) -> ContentDisposition? { + guard let value = value?.lowercased() else { return nil } + if value.hasPrefix("inline") { return .inline } + if value.hasPrefix("attachment") { return .attachment } + return nil + } + + private static func extractParameter(_ headerValue: String, name: String) -> String? { + let lower = headerValue.lowercased() + let search = "\(name.lowercased())=" + guard let range = lower.range(of: search) else { return nil } + var value = String(headerValue[range.upperBound...]) + if value.hasPrefix("\"") { + value = String(value.dropFirst()) + if let endQuote = value.firstIndex(of: "\"") { + value = String(value[.. String? { + guard var cid = value?.trimmingCharacters(in: .whitespaces) else { return nil } + if cid.hasPrefix("<") { cid = String(cid.dropFirst()) } + if cid.hasSuffix(">") { cid = String(cid.dropLast()) } + return cid.isEmpty ? nil : cid + } + + private static func estimateDecodedSize(_ part: MIMEPart) -> Int { + if part.transferEncoding == .base64 { + // base64 inflates ~33%: decoded = encoded * 3 / 4 + let base64Length = part.body.count + // But body is already decoded at this point, so use body.count directly + return part.body.count + } + return part.body.count + } + + private static func decodeQuotedPrintable(_ input: String) -> Data { + var data = Data() + let lines = input.components(separatedBy: "\n") + + for (lineIndex, line) in lines.enumerated() { + var processedLine = line + if processedLine.hasSuffix("\r") { + processedLine = String(processedLine.dropLast()) + } + + // Check for soft line break + if processedLine.hasSuffix("=") { + processedLine = String(processedLine.dropLast()) + data.append(contentsOf: decodeQPLine(processedLine)) + } else { + data.append(contentsOf: decodeQPLine(processedLine)) + if lineIndex < lines.count - 1 { + data.append(contentsOf: "\r\n".utf8) + } + } + } + + return data + } + + private static func decodeQPLine(_ line: String) -> Data { + var data = Data() + var i = line.startIndex + while i < line.endIndex { + if line[i] == "=" { + let next1 = line.index(after: i) + guard next1 < line.endIndex else { + data.append(contentsOf: "=".utf8) + break + } + let next2 = line.index(after: next1) + guard next2 < line.endIndex else { + data.append(contentsOf: String(line[i...]).utf8) + break + } + let hex = String(line[next1...next2]) + if let byte = UInt8(hex, radix: 16) { + data.append(byte) + i = line.index(after: next2) + } else { + data.append(contentsOf: "=".utf8) + i = next1 + } + } else { + data.append(contentsOf: String(line[i]).utf8) + i = line.index(after: i) + } + } + return data + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MIMEParserTests 2>&1 | tail -10 +``` + +Expected: All 11 tests pass. + +- [ ] **Step 5: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: 111 + 6 + 11 = 128 tests pass (existing + RFC2047 + MIMEParser). + +- [ ] **Step 6: Commit** + +```bash +git add Sources/MIMEParser/MIMEParser.swift Tests/MIMEParserTests/MIMEParserTests.swift +git commit -m "add MIMEParser: multipart parsing, content decoding, boundary generation" +``` + +--- + +## Chunk 2: Schema Migration, Store Updates & IMAPClient Extensions + +### Task 5: v4_attachment migration + +**Files:** +- Modify: `Sources/MailStore/DatabaseSetup.swift` +- Modify: `Sources/MailStore/Records/AttachmentRecord.swift` +- Modify: `Sources/MailStore/Records/MessageRecord.swift` +- Modify: `Tests/MailStoreTests/MigrationTests.swift` + +- [ ] **Step 1: Write failing migration test** + +Open `Tests/MailStoreTests/MigrationTests.swift` and add a test to the existing suite: + +```swift +@Test("v4_attachment migration adds sectionPath to attachment and hasAttachments to message") +func v4AttachmentMigration() throws { + let db = try DatabaseSetup.openInMemoryDatabase() + try db.read { db in + let attachmentColumns = try db.columns(in: "attachment") + let attachmentColumnNames = attachmentColumns.map(\.name) + #expect(attachmentColumnNames.contains("sectionPath")) + + let messageColumns = try db.columns(in: "message") + let messageColumnNames = messageColumns.map(\.name) + #expect(messageColumnNames.contains("hasAttachments")) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MigrationTests/v4AttachmentMigration 2>&1 | tail -5 +``` + +Expected: FAIL — columns don't exist yet. + +- [ ] **Step 3: Add migration to DatabaseSetup.swift** + +Add before the `return migrator` line: + +```swift +migrator.registerMigration("v4_attachment") { db in + try db.alter(table: "attachment") { t in + t.add(column: "sectionPath", .text) + } + try db.alter(table: "message") { t in + t.add(column: "hasAttachments", .boolean).notNull().defaults(to: false) + } +} +``` + +- [ ] **Step 4: Update AttachmentRecord to include sectionPath** + +Add `sectionPath` property to `AttachmentRecord`: + +```swift +public struct AttachmentRecord: Codable, FetchableRecord, PersistableRecord, Sendable { + public static let databaseTableName = "attachment" + + public var id: String + public var messageId: String + public var filename: String? + public var mimeType: String + public var size: Int + public var contentId: String? + public var cachePath: String? + public var sectionPath: String? + + public init( + id: String, messageId: String, filename: String?, mimeType: String, + size: Int, contentId: String?, cachePath: String?, sectionPath: String? = nil + ) { + self.id = id + self.messageId = messageId + self.filename = filename + self.mimeType = mimeType + self.size = size + self.contentId = contentId + self.cachePath = cachePath + self.sectionPath = sectionPath + } +} +``` + +- [ ] **Step 5: Update MessageRecord to include hasAttachments** + +Add `hasAttachments` property to `MessageRecord`. Add it after `size`: + +```swift +public var hasAttachments: Bool +``` + +Update the init to include the new parameter with a default value: + +```swift +public init( + id: String, accountId: String, mailboxId: String, uid: Int, + messageId: String?, inReplyTo: String?, refs: String?, + subject: String?, fromAddress: String?, fromName: String?, + toAddresses: String?, ccAddresses: String?, + date: String, snippet: String?, bodyText: String?, bodyHtml: String?, + isRead: Bool, isFlagged: Bool, size: Int, hasAttachments: Bool = false +) { + // ... existing assignments ... + self.hasAttachments = hasAttachments +} +``` + +- [ ] **Step 6: Run test to verify it passes** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MigrationTests 2>&1 | tail -5 +``` + +Expected: All migration tests pass. + +- [ ] **Step 7: Run full test suite to check nothing broke** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. The new `hasAttachments` field has a default value of `false`, so existing code continues to work. + +- [ ] **Step 8: Commit** + +```bash +git add Sources/MailStore/DatabaseSetup.swift Sources/MailStore/Records/AttachmentRecord.swift Sources/MailStore/Records/MessageRecord.swift Tests/MailStoreTests/MigrationTests.swift +git commit -m "add v4_attachment migration: sectionPath on attachment, hasAttachments on message" +``` + +--- + +### Task 6: MailStore attachment methods & update Queries.swift + +**Files:** +- Modify: `Sources/MailStore/MailStore.swift` +- Modify: `Sources/MailStore/Queries.swift` + +- [ ] **Step 1: Add attachment CRUD methods to MailStore** + +Add to MailStore.swift in a new `// MARK: - Attachments` section: + +```swift +// MARK: - Attachments + +public func insertAttachment(_ attachment: AttachmentRecord) throws { + try dbWriter.write { db in + try attachment.insert(db) + } +} + +public func insertAttachments(_ attachments: [AttachmentRecord]) throws { + try dbWriter.write { db in + for attachment in attachments { + try attachment.insert(db) + } + } +} + +public func attachments(messageId: String) throws -> [AttachmentRecord] { + try dbWriter.read { db in + try AttachmentRecord + .filter(Column("messageId") == messageId) + .order(Column("filename")) + .fetchAll(db) + } +} + +public func updateAttachmentCachePath(id: String, cachePath: String) throws { + try dbWriter.write { db in + try db.execute( + sql: "UPDATE attachment SET cachePath = ? WHERE id = ?", + arguments: [cachePath, id] + ) + } +} + +public func updateHasAttachments(messageId: String, hasAttachments: Bool) throws { + try dbWriter.write { db in + try db.execute( + sql: "UPDATE message SET hasAttachments = ? WHERE id = ?", + arguments: [hasAttachments, messageId] + ) + } +} + +public func storeBodyWithAttachments( + messageId: String, + text: String?, + html: String?, + attachments: [AttachmentRecord] +) throws { + try dbWriter.write { db in + try db.execute( + sql: "UPDATE message SET bodyText = ?, bodyHtml = ?, hasAttachments = ? WHERE id = ?", + arguments: [text, html, !attachments.isEmpty, messageId] + ) + for attachment in attachments { + try attachment.insert(db) + } + } +} +``` + +- [ ] **Step 2: Update Queries.swift to read hasAttachments** + +In `Queries.swift`, update `toMessageSummary` to read from the record: + +Change: +```swift +hasAttachments: false +``` +To: +```swift +hasAttachments: record.hasAttachments +``` + +- [ ] **Step 3: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add Sources/MailStore/MailStore.swift Sources/MailStore/Queries.swift +git commit -m "add attachment CRUD to MailStore, wire hasAttachments in Queries" +``` + +--- + +### Task 7: IMAPClient protocol & implementation — fetchFullMessage, fetchSection + +**Files:** +- Modify: `Sources/IMAPClient/IMAPClientProtocol.swift` +- Modify: `Sources/IMAPClient/IMAPClient.swift` +- Modify: `Tests/SyncEngineTests/MockIMAPClient.swift` + +- [ ] **Step 1: Add new methods to IMAPClientProtocol** + +Add to `IMAPClientProtocol.swift`: + +```swift +// v0.5 attachment/MIME operations +func fetchFullMessage(uid: Int) async throws -> String +func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data +``` + +- [ ] **Step 2: Implement fetchFullMessage in IMAPClient** + +Add to `IMAPClient.swift` in the `// MARK: - Fetch operations` section: + +```swift +public func fetchFullMessage(uid: Int) async throws -> String { + guard var runner else { throw IMAPError.notConnected } + let uidValue = UID(rawValue: UInt32(uid)) + let range = MessageIdentifierRange(uidValue...uidValue) + let set = MessageIdentifierSetNonEmpty(range: range) + let responses = try await runner.run(.uidFetch( + .set(set), + [.bodySection(peek: true, SectionSpecifier(kind: .complete), nil)], + [] + )) + self.runner = runner + return parseFullMessageResponse(responses) +} +``` + +Add the parsing helper: + +```swift +private func parseFullMessageResponse(_ responses: [Response]) -> String { + var bodyBuffer = ByteBuffer() + for response in responses { + if case .fetch(let fetchResponse) = response { + switch fetchResponse { + case .streamingBytes(let bytes): + var mutableBytes = bytes + bodyBuffer.writeBuffer(&mutableBytes) + default: + break + } + } + } + return String(buffer: bodyBuffer) +} +``` + +- [ ] **Step 3: Implement fetchSection in IMAPClient** + +Add to `IMAPClient.swift`: + +```swift +public func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data { + guard var runner else { throw IMAPError.notConnected } + + // Select the mailbox first + let selectResponses = try await runner.run(.select(MailboxName(ByteBuffer(string: mailbox)))) + self.runner = runner + guard selectResponses.contains(where: { isOKTagged($0) }) else { + throw IMAPError.unexpectedResponse("SELECT \(mailbox) failed") + } + + let uidValue = UID(rawValue: UInt32(uid)) + let range = MessageIdentifierRange(uidValue...uidValue) + let set = MessageIdentifierSetNonEmpty(range: range) + + // Parse section path into SectionSpecifier.Part + let sectionParts = section.split(separator: ".").compactMap { Int($0) } + let part = SectionSpecifier.Part(sectionParts) + let spec = SectionSpecifier(part: part) + + let responses = try await runner.run(.uidFetch( + .set(set), + [.bodySection(peek: true, spec, nil)], + [] + )) + self.runner = runner + + var bodyBuffer = ByteBuffer() + for response in responses { + if case .fetch(let fetchResponse) = response { + if case .streamingBytes(let bytes) = fetchResponse { + var mutableBytes = bytes + bodyBuffer.writeBuffer(&mutableBytes) + } + } + } + + // The section content may be base64 encoded — decode it + let raw = String(buffer: bodyBuffer) + let cleaned = raw.filter { !$0.isWhitespace } + if let decoded = Data(base64Encoded: cleaned) { + return decoded + } + return Data(bodyBuffer.readableBytesView) +} +``` + +- [ ] **Step 4: Update MockIMAPClient** + +Add to `MockIMAPClient.swift`: + +Properties: +```swift +var fullMessages: [Int: String] = [:] +var sections: [String: Data] = [:] // key: "uid-section" +var fetchFullMessageCalls: [Int] = [] +var fetchSectionCalls: [(uid: Int, mailbox: String, section: String)] = [] +``` + +Methods: +```swift +func fetchFullMessage(uid: Int) async throws -> String { + fetchFullMessageCalls.append(uid) + return fullMessages[uid] ?? "" +} + +func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data { + fetchSectionCalls.append((uid: uid, mailbox: mailbox, section: section)) + return sections["\(uid)-\(section)"] ?? Data() +} +``` + +- [ ] **Step 5: Build to verify compilation** + +```bash +cd Packages/MagnumOpusCore +swift build +``` + +Expected: Build succeeds. + +- [ ] **Step 6: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. + +- [ ] **Step 7: Commit** + +```bash +git add Sources/IMAPClient/IMAPClientProtocol.swift Sources/IMAPClient/IMAPClient.swift Tests/SyncEngineTests/MockIMAPClient.swift +git commit -m "add fetchFullMessage, fetchSection to IMAPClient for MIME attachment retrieval" +``` + +--- + +## Chunk 3: MessageFormatter Multipart & SyncCoordinator MIME Integration + +### Task 8: MessageFormatter — multipart formatting & base64 encoding + +**Files:** +- Modify: `Sources/SMTPClient/MessageFormatter.swift` +- Modify: `Tests/SMTPClientTests/MessageFormatterTests.swift` + +- [ ] **Step 1: Write failing tests for multipart formatting** + +Add to `MessageFormatterTests.swift`: + +```swift +// MARK: - Multipart Formatting + +@Test("format with attachments produces multipart/mixed") +func multipartWithAttachment() { + let message = OutgoingMessage( + from: EmailAddress(address: "alice@example.com"), + to: [EmailAddress(address: "bob@example.com")], + subject: "With attachment", + bodyText: "See attached.", + messageId: "test@example.com" + ) + let attachments: [(filename: String, mimeType: String, data: Data)] = [ + (filename: "test.pdf", mimeType: "application/pdf", data: Data("PDF content".utf8)), + ] + + let formatted = MessageFormatter.formatMultipart(message, attachments: attachments) + + #expect(formatted.contains("Content-Type: multipart/mixed; boundary=")) + #expect(formatted.contains("Content-Type: text/plain; charset=utf-8")) + #expect(formatted.contains("Content-Type: application/pdf; name=\"test.pdf\"")) + #expect(formatted.contains("Content-Disposition: attachment; filename=\"test.pdf\"")) + #expect(formatted.contains("Content-Transfer-Encoding: base64")) + #expect(formatted.contains("See attached.")) +} + +@Test("format without attachments produces single-part") +func singlePartNoAttachments() { + let message = OutgoingMessage( + from: EmailAddress(address: "alice@example.com"), + to: [EmailAddress(address: "bob@example.com")], + subject: "Plain", + bodyText: "Just text.", + messageId: "test@example.com" + ) + + let formatted = MessageFormatter.format(message) + + #expect(formatted.contains("Content-Type: text/plain; charset=utf-8")) + #expect(!formatted.contains("multipart")) +} + +@Test("base64 encoding wraps at 76 characters") +func base64LineWrapping() { + let data = Data(repeating: 0xFF, count: 100) + let encoded = MessageFormatter.base64Encode(data) + let lines = encoded.components(separatedBy: "\r\n") + for line in lines where !line.isEmpty { + #expect(line.count <= 76) + } +} + +@Test("multiple attachments produce correct number of boundary sections") +func multipleAttachments() { + let message = OutgoingMessage( + from: EmailAddress(address: "alice@example.com"), + to: [EmailAddress(address: "bob@example.com")], + subject: "Multi", + bodyText: "Files.", + messageId: "test@example.com" + ) + let attachments: [(filename: String, mimeType: String, data: Data)] = [ + (filename: "a.pdf", mimeType: "application/pdf", data: Data("A".utf8)), + (filename: "b.jpg", mimeType: "image/jpeg", data: Data("B".utf8)), + ] + + let formatted = MessageFormatter.formatMultipart(message, attachments: attachments) + + #expect(formatted.contains("a.pdf")) + #expect(formatted.contains("b.jpg")) + #expect(formatted.contains("application/pdf")) + #expect(formatted.contains("image/jpeg")) +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MessageFormatterTests 2>&1 | tail -5 +``` + +Expected: Compilation error — `formatMultipart` and `base64Encode` not defined. + +- [ ] **Step 3: Implement formatMultipart and base64Encode** + +Add to `MessageFormatter.swift`: + +```swift +/// Format a multipart/mixed message with attachments. +public static func formatMultipart( + _ message: OutgoingMessage, + attachments: [(filename: String, mimeType: String, data: Data)] +) -> String { + let boundary = "=_MagnumOpus_\(UUID().uuidString)" + + var headers: [(String, String)] = [] + headers.append(("From", formatAddress(message.from))) + headers.append(("To", message.to.map(formatAddress).joined(separator: ", "))) + + if !message.cc.isEmpty { + headers.append(("Cc", message.cc.map(formatAddress).joined(separator: ", "))) + } + + headers.append(("Subject", message.subject)) + headers.append(("Date", formatRFC2822Date(Date()))) + headers.append(("Message-ID", "<\(message.messageId)>")) + + if let inReplyTo = message.inReplyTo { + headers.append(("In-Reply-To", "<\(inReplyTo)>")) + } + if let references = message.references { + headers.append(("References", references)) + } + + headers.append(("MIME-Version", "1.0")) + headers.append(("Content-Type", "multipart/mixed; boundary=\"\(boundary)\"")) + + var result = "" + for (name, value) in headers { + result += "\(name): \(value)\r\n" + } + result += "\r\n" + + // Text body part + result += "--\(boundary)\r\n" + result += "Content-Type: text/plain; charset=utf-8\r\n" + result += "Content-Transfer-Encoding: quoted-printable\r\n" + result += "\r\n" + result += quotedPrintableEncode(message.bodyText) + result += "\r\n" + + // Attachment parts + for attachment in attachments { + result += "--\(boundary)\r\n" + result += "Content-Type: \(attachment.mimeType); name=\"\(attachment.filename)\"\r\n" + result += "Content-Disposition: attachment; filename=\"\(attachment.filename)\"\r\n" + result += "Content-Transfer-Encoding: base64\r\n" + result += "\r\n" + result += base64Encode(attachment.data) + result += "\r\n" + } + + // Closing boundary + result += "--\(boundary)--\r\n" + + return result +} + +/// Base64 encode data with line wrapping at 76 characters per RFC 2045. +public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String { + let encoded = data.base64EncodedString() + var result = "" + var index = encoded.startIndex + while index < encoded.endIndex { + let end = encoded.index(index, offsetBy: lineLength, limitedBy: encoded.endIndex) ?? encoded.endIndex + result += String(encoded[index..&1 | tail -5 +``` + +Expected: All tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add Sources/SMTPClient/MessageFormatter.swift Tests/SMTPClientTests/MessageFormatterTests.swift +git commit -m "add multipart/mixed formatting, base64 line-wrapped encoding to MessageFormatter" +``` + +--- + +### Task 9: SyncCoordinator MIME-aware body prefetch + +**Files:** +- Modify: `Sources/SyncEngine/SyncCoordinator.swift` +- Modify: `Package.swift` (add MIMEParser dependency to SyncEngine) +- Modify: `Tests/SyncEngineTests/SyncCoordinatorTests.swift` + +- [ ] **Step 1: Add MIMEParser dependency to SyncEngine target** + +In `Package.swift`, update the SyncEngine target dependencies: + +```swift +.target( + name: "SyncEngine", + dependencies: ["Models", "IMAPClient", "SMTPClient", "MailStore", "TaskStore", "MIMEParser"] +), +``` + +- [ ] **Step 2: Write failing test for MIME-aware sync** + +Add to `SyncCoordinatorTests.swift`: + +```swift +@Test("sync with full message parses MIME and stores attachments") +func mimeAwareSync() async throws { + let store = try makeStore() + let mock = makeMock() + + // Provide a multipart MIME message for body prefetch + let mimeMessage = """ + Content-Type: multipart/mixed; boundary="bound"\r + \r + --bound\r + Content-Type: text/plain; charset=utf-8\r + \r + Hello from MIME.\r + --bound\r + Content-Type: application/pdf; name="report.pdf"\r + Content-Disposition: attachment; filename="report.pdf"\r + Content-Transfer-Encoding: base64\r + \r + SGVsbG8=\r + --bound-- + """ + mock.fullMessages[1] = mimeMessage + + let coordinator = SyncCoordinator( + accountConfig: AccountConfig( + id: "acc1", name: "Personal", email: "me@example.com", + imapHost: "imap.example.com", imapPort: 993 + ), + imapClient: mock, + store: store + ) + + try await coordinator.syncNow() + + // Verify MIME body was stored + let inboxMb = try store.mailboxes(accountId: "acc1").first { $0.name == "INBOX" }! + let messages = try store.messages(mailboxId: inboxMb.id) + let msg = messages.first { $0.uid == 1 }! + #expect(msg.bodyText == "Hello from MIME.") + #expect(msg.hasAttachments == true) + + // Verify attachment was stored + let attachments = try store.attachments(messageId: msg.id) + #expect(attachments.count == 1) + #expect(attachments.first?.filename == "report.pdf") + #expect(attachments.first?.mimeType == "application/pdf") + #expect(attachments.first?.sectionPath == "2") +} +``` + +- [ ] **Step 3: Run test to verify it fails** + +```bash +cd Packages/MagnumOpusCore +swift test --filter SyncCoordinatorTests/mimeAwareSync 2>&1 | tail -5 +``` + +Expected: FAIL — `fullMessages` not in mock or body not parsed as MIME. + +- [ ] **Step 4: Update SyncCoordinator.prefetchBodies to use MIME parsing** + +Replace the `prefetchBodies` method in `SyncCoordinator.swift`: + +```swift +import MIMEParser +``` + +(Add at the top of the file with other imports.) + +```swift +/// Fetch full RFC822 messages and parse MIME for body + attachments +private func prefetchBodies(mailboxId: String) async { + let thirtyDaysAgo = ISO8601DateFormatter().string( + from: Calendar.current.date(byAdding: .day, value: -30, to: Date())! + ) + do { + let messages = try store.messages(mailboxId: mailboxId) + let recent = messages.filter { $0.bodyText == nil && $0.bodyHtml == nil && $0.date >= thirtyDaysAgo } + for message in recent.prefix(50) { + guard !Task.isCancelled else { break } + let rawMessage = try await imapClient.fetchFullMessage(uid: message.uid) + guard !rawMessage.isEmpty else { continue } + + let parsed = MIMEParser.parse(rawMessage) + + // Build attachment records + let attachmentRecords = (parsed.attachments + parsed.inlineImages).map { att in + AttachmentRecord( + id: UUID().uuidString, + messageId: message.id, + filename: att.filename, + mimeType: att.mimeType, + size: att.size, + contentId: att.contentId, + cachePath: nil, + sectionPath: att.sectionPath + ) + } + + try store.storeBodyWithAttachments( + messageId: message.id, + text: parsed.textBody, + html: parsed.htmlBody, + attachments: attachmentRecords + ) + } + } catch { + // Background prefetch failure is non-fatal + print("[SyncCoordinator] prefetchBodies error: \(error)") + } +} +``` + +- [ ] **Step 5: Run test to verify it passes** + +```bash +cd Packages/MagnumOpusCore +swift test --filter SyncCoordinatorTests/mimeAwareSync 2>&1 | tail -5 +``` + +Expected: PASS. + +- [ ] **Step 6: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. + +- [ ] **Step 7: Commit** + +```bash +git add Package.swift Sources/SyncEngine/SyncCoordinator.swift Tests/SyncEngineTests/SyncCoordinatorTests.swift +git commit -m "replace body text prefetch with MIME-aware parsing, store attachments from parsed messages" +``` + +--- + +## Chunk 4: IMAP IDLE + +### Task 10: IMAPIdleHandler — NIO channel handler for IDLE responses + +**Files:** +- Create: `Sources/IMAPClient/IMAPIdleHandler.swift` + +- [ ] **Step 1: Write IMAPIdleHandler** + +```swift +import NIO +import NIOIMAPCore +import NIOIMAP + +/// Events emitted by the IDLE handler +public enum IMAPIdleEvent: Sendable { + case exists(Int) + case expunge(Int) + case idleTerminated +} + +/// NIO ChannelInboundHandler that processes untagged responses during IMAP IDLE. +/// Unlike the standard IMAPResponseHandler (which uses CheckedContinuation for tagged responses), +/// this handler uses AsyncStream to deliver a continuous stream of events. +final class IMAPIdleHandler: ChannelInboundHandler, @unchecked Sendable { + typealias InboundIn = Response + + private let continuation: AsyncStream.Continuation + private var idleTag: String? + + init(continuation: AsyncStream.Continuation) { + self.continuation = continuation + } + + func setIdleTag(_ tag: String) { + idleTag = tag + } + + func channelRead(context: ChannelHandlerContext, data: NIOAny) { + let response = unwrapInboundIn(data) + + switch response { + case .untagged(let payload): + switch payload { + case .mailboxData(let data): + switch data { + case .exists(let count): + continuation.yield(.exists(count)) + default: + break + } + case .messageData(let data): + switch data { + case .expunge(let seqNum): + continuation.yield(.expunge(seqNum)) + default: + break + } + default: + break + } + case .tagged(let tagged): + if tagged.tag == idleTag { + continuation.yield(.idleTerminated) + } + case .idleStarted: + // Server acknowledged IDLE — we're now idling + break + case .fetch, .authenticationChallenge, .fatal: + break + } + } + + func errorCaught(context: ChannelHandlerContext, error: Error) { + continuation.finish() + context.close(promise: nil) + } + + func channelInactive(context: ChannelHandlerContext) { + continuation.finish() + } +} +``` + +- [ ] **Step 2: Verify build** + +```bash +cd Packages/MagnumOpusCore +swift build +``` + +Expected: Build succeeds. + +- [ ] **Step 3: Commit** + +```bash +git add Sources/IMAPClient/IMAPIdleHandler.swift +git commit -m "add IMAPIdleHandler: NIO channel handler for IDLE event streaming" +``` + +--- + +### Task 11: IMAPIdleClient actor + +**Files:** +- Create: `Sources/IMAPClient/IMAPIdleClient.swift` +- Create: `Tests/IMAPClientTests/IMAPIdleClientTests.swift` + +- [ ] **Step 1: Write IMAPIdleClient** + +```swift +import Foundation +import NIO +import NIOIMAPCore +@preconcurrency import NIOIMAP +@preconcurrency import NIOSSL +import Models + +public actor IMAPIdleClient { + private let host: String + private let port: Int + private let credentials: Credentials + private var channel: Channel? + private var group: EventLoopGroup? + private var isMonitoring = false + private var monitorTask: Task? + private let reIdleInterval: Duration = .seconds(29 * 60) // 29 minutes per RFC 2177 + + public init(host: String, port: Int, credentials: Credentials) { + self.host = host + self.port = port + self.credentials = credentials + } + + /// Start monitoring INBOX via IMAP IDLE. Calls onNewMail when server sends EXISTS. + public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws { + guard !isMonitoring else { return } + isMonitoring = true + + try await connectAndLogin() + try await selectInbox() + + monitorTask = Task { [weak self] in + var backoffSeconds: UInt64 = 5 + while !Task.isCancelled { + guard let self else { break } + do { + try await self.idleLoop(onNewMail: onNewMail) + } catch { + if Task.isCancelled { break } + // Reconnect with exponential backoff + try? await Task.sleep(for: .seconds(Int(backoffSeconds))) + backoffSeconds = min(backoffSeconds * 2, 300) // cap at 5 min + do { + try await self.connectAndLogin() + try await self.selectInbox() + backoffSeconds = 5 // reset on success + } catch { + if Task.isCancelled { break } + continue + } + } + } + } + } + + /// Stop monitoring and disconnect. + public func stopMonitoring() async { + isMonitoring = false + monitorTask?.cancel() + monitorTask = nil + try? await channel?.close() + channel = nil + try? await group?.shutdownGracefully() + group = nil + } + + // MARK: - Connection + + private func connectAndLogin() async throws { + let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1) + let sslContext = try NIOSSLContext(configuration: TLSConfiguration.makeClientConfiguration()) + let hostname = host + + let responseHandler = IMAPResponseHandler() + + let bootstrap = ClientBootstrap(group: eventLoopGroup) + .channelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1) + .channelInitializer { channel in + let sslHandler = try! NIOSSLClientHandler(context: sslContext, serverHostname: hostname) + return channel.pipeline.addHandlers([ + sslHandler, + IMAPClientHandler(), + responseHandler, + ]) + } + + let chan = try await bootstrap.connect(host: host, port: port).get() + try await responseHandler.waitForGreeting() + + // Login + var tagCounter = 0 + func nextTag() -> String { + tagCounter += 1 + return "IDLE\(tagCounter)" + } + + let loginTag = nextTag() + let loginCommand = TaggedCommand( + tag: loginTag, + command: .login(username: credentials.username, password: credentials.password) + ) + let loginResponses = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[Response], Error>) in + responseHandler.sendCommand(tag: loginTag, continuation: cont) + chan.writeAndFlush(IMAPClientHandler.Message.part(.tagged(loginCommand)), promise: nil) + } + guard loginResponses.contains(where: { isOK($0) }) else { + throw IMAPError.authenticationFailed + } + + self.channel = chan + self.group = eventLoopGroup + } + + private func selectInbox() async throws { + guard let channel else { throw IMAPError.notConnected } + + let responseHandler = try await getResponseHandler() + let tag = "IDLESEL1" + let selectCommand = TaggedCommand( + tag: tag, + command: .select(MailboxName(ByteBuffer(string: "INBOX"))) + ) + let responses = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[Response], Error>) in + responseHandler.sendCommand(tag: tag, continuation: cont) + channel.writeAndFlush(IMAPClientHandler.Message.part(.tagged(selectCommand)), promise: nil) + } + guard responses.contains(where: { isOK($0) }) else { + throw IMAPError.unexpectedResponse("SELECT INBOX failed") + } + } + + // MARK: - IDLE Loop + + private func idleLoop(onNewMail: @escaping @Sendable () -> Void) async throws { + guard let channel else { throw IMAPError.notConnected } + let pipeline = channel.pipeline + + // Iterative IDLE loop — avoids unbounded stack growth from recursion + while !Task.isCancelled { + // Swap response handler for IDLE handler + let (stream, streamContinuation) = AsyncStream.makeStream() + let idleHandler = IMAPIdleHandler(continuation: streamContinuation) + + let oldHandler = try await getResponseHandler() + try await pipeline.removeHandler(oldHandler).get() + try await pipeline.addHandler(idleHandler).get() + + let idleTag = "IDLE1" + idleHandler.setIdleTag(idleTag) + + // Send IDLE command + let idleCommand = TaggedCommand(tag: idleTag, command: .idleStart) + channel.writeAndFlush(IMAPClientHandler.Message.part(.tagged(idleCommand)), promise: nil) + + // Set up re-IDLE timer — after 29 min, send DONE to break IDLE (RFC 2177) + let reIdleChannel = channel + let reIdleContinuation = streamContinuation + let reIdleTask = Task { + try await Task.sleep(for: reIdleInterval) + // Timer fired — break IDLE so the outer loop re-enters + reIdleChannel.writeAndFlush(IMAPClientHandler.Message.part(.idleDone), promise: nil) + } + + var shouldReIdle = false + + // Consume events + for await event in stream { + switch event { + case .exists: + // New mail — break IDLE, trigger sync + channel.writeAndFlush(IMAPClientHandler.Message.part(.idleDone), promise: nil) + for await innerEvent in stream { + if case .idleTerminated = innerEvent { break } + } + reIdleTask.cancel() + onNewMail() + shouldReIdle = true + + case .expunge: + break + + case .idleTerminated: + reIdleTask.cancel() + shouldReIdle = true + } + if shouldReIdle { break } + } + + // Restore response handler before re-entering IDLE + try await pipeline.removeHandler(idleHandler).get() + let newResponseHandler = IMAPResponseHandler() + try await pipeline.addHandler(newResponseHandler).get() + + if !shouldReIdle { + // Stream ended — connection dropped + throw IMAPError.serverError("IDLE connection dropped") + } + // Otherwise: loop back to re-enter IDLE + } + } + + // MARK: - Helpers + + private func getResponseHandler() async throws -> IMAPResponseHandler { + guard let channel else { throw IMAPError.notConnected } + return try await channel.pipeline.handler(type: IMAPResponseHandler.self).get() + } + + private func isOK(_ response: Response) -> Bool { + if case .tagged(let tagged) = response { + if case .ok = tagged.state { return true } + } + return false + } +} +``` + +- [ ] **Step 2: Write basic IDLE client tests** + +```swift +import Testing +import Foundation +@testable import IMAPClient +import Models + +@Suite("IMAPIdleClient") +struct IMAPIdleClientTests { + + @Test("IMAPIdleClient can be initialized") + func initialization() { + let client = IMAPIdleClient( + host: "imap.example.com", + port: 993, + credentials: Credentials(username: "user", password: "pass") + ) + // Just verify it compiles and initializes — actual IDLE testing + // requires a real server or more sophisticated mocking + #expect(true) + } + + @Test("IMAPIdleEvent cases exist") + func eventCases() { + let exists = IMAPIdleEvent.exists(42) + let expunge = IMAPIdleEvent.expunge(1) + let terminated = IMAPIdleEvent.idleTerminated + + if case .exists(let count) = exists { + #expect(count == 42) + } + if case .expunge(let num) = expunge { + #expect(num == 1) + } + if case .idleTerminated = terminated { + #expect(true) + } + } +} +``` + +- [ ] **Step 3: Run tests** + +```bash +cd Packages/MagnumOpusCore +swift test --filter IMAPIdleClientTests 2>&1 | tail -5 +``` + +Expected: Tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add Sources/IMAPClient/IMAPIdleClient.swift Tests/IMAPClientTests/IMAPIdleClientTests.swift +git commit -m "add IMAPIdleClient actor with IDLE loop, reconnect backoff, re-IDLE timer" +``` + +--- + +### Task 12: SyncCoordinator IDLE integration + +**Files:** +- Modify: `Sources/SyncEngine/SyncCoordinator.swift` +- Modify: `Tests/SyncEngineTests/SyncCoordinatorTests.swift` + +- [ ] **Step 1: Write failing test for IDLE integration** + +Add to `SyncCoordinatorTests.swift`: + +```swift +@Test("startIdleMonitoring does nothing when server lacks IDLE capability") +func idleNotSupported() async throws { + let store = try makeStore() + let mock = makeMock() + mock.serverCapabilities = ["IMAP4rev1"] // No IDLE + + let coordinator = SyncCoordinator( + accountConfig: AccountConfig( + id: "acc1", name: "Personal", email: "me@example.com", + imapHost: "imap.example.com", imapPort: 993 + ), + imapClient: mock, + store: store + ) + + // First sync to establish account + try await coordinator.syncNow() + + // Should not crash, just return without starting IDLE + await coordinator.startIdleMonitoring() + + // Verify no IDLE client was created (by checking that sync still works normally) + try await coordinator.syncNow() + #expect(coordinator.syncState == .idle) +} + +@Test("stopIdleMonitoring is safe to call even when not monitoring") +func stopIdleWhenNotMonitoring() async throws { + let store = try makeStore() + let mock = makeMock() + let coordinator = SyncCoordinator( + accountConfig: AccountConfig( + id: "acc1", name: "Personal", email: "me@example.com", + imapHost: "imap.example.com", imapPort: 993 + ), + imapClient: mock, + store: store + ) + + // Should not crash + await coordinator.stopIdleMonitoring() + #expect(coordinator.syncState == .idle) +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +```bash +cd Packages/MagnumOpusCore +swift test --filter "SyncCoordinatorTests/idleNotSupported|SyncCoordinatorTests/stopIdleWhenNotMonitoring" 2>&1 | tail -5 +``` + +Expected: Compilation error — methods don't exist. + +- [ ] **Step 3: Add IDLE integration to SyncCoordinator** + +First, update the `SyncCoordinator` init to accept `Credentials` for IDLE use. Add a new stored property: + +```swift +private let credentials: Credentials? +``` + +Update the init signature (add `credentials` parameter with default `nil`): + +```swift +public init( + accountConfig: AccountConfig, + imapClient: any IMAPClientProtocol, + store: MailStore, + actionQueue: ActionQueue? = nil, + taskStore: TaskStore? = nil, + credentials: Credentials? = nil +) { + self.accountConfig = accountConfig + self.imapClient = imapClient + self.store = store + self.actionQueue = actionQueue + self.taskStore = taskStore + self.credentials = credentials +} +``` + +Add IDLE properties after existing ones: + +```swift +private var idleClient: IMAPIdleClient? +private var idleActive = false +``` + +Add IDLE methods: + +```swift +// MARK: - IMAP IDLE + +/// Check server capabilities and start IDLE monitoring if supported. +/// Must be called after at least one successful sync (so capabilities are cached). +public func startIdleMonitoring() async { + guard let credentials else { + print("[SyncCoordinator] No credentials provided, cannot start IDLE") + return + } + + do { + let caps = try await imapClient.capabilities() + guard caps.contains("IDLE") else { + print("[SyncCoordinator] Server does not support IDLE, using periodic sync only") + return + } + + let client = IMAPIdleClient( + host: accountConfig.imapHost, + port: accountConfig.imapPort, + credentials: credentials + ) + + idleClient = client + idleActive = true + + try await client.startMonitoring { [weak self] in + Task { @MainActor [weak self] in + try? await self?.syncNow() + } + } + } catch { + print("[SyncCoordinator] Failed to start IDLE monitoring: \(error)") + } +} + +/// Stop IDLE monitoring and clean up. +public func stopIdleMonitoring() async { + idleActive = false + await idleClient?.stopMonitoring() + idleClient = nil +} +``` + +**Note:** Existing callers that don't pass `credentials` continue to work — `credentials` defaults to `nil` and IDLE simply won't start. The app layer passes credentials when constructing SyncCoordinator. + +- [ ] **Step 4: Run tests to verify they pass** + +```bash +cd Packages/MagnumOpusCore +swift test --filter SyncCoordinatorTests 2>&1 | tail -5 +``` + +Expected: All SyncCoordinator tests pass. + +- [ ] **Step 5: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. + +- [ ] **Step 6: Commit** + +```bash +git add Sources/SyncEngine/SyncCoordinator.swift Tests/SyncEngineTests/SyncCoordinatorTests.swift +git commit -m "add IDLE monitoring to SyncCoordinator: start/stop with capability check" +``` + +--- + +## Chunk 5: SyncEngine Update — Stop Fetching Body in Envelopes + +### Task 13: Remove body fetch from fetchEnvelopes + +The existing `fetchEnvelopes` fetches body text inline during initial sync via `BODY[TEXT]`. Now that `prefetchBodies` uses `fetchFullMessage` with MIME parsing, we should stop fetching bodies in `fetchEnvelopes` to avoid doubling bandwidth. The body data from envelopes was being stored as raw text without MIME parsing anyway, so it's better to let the MIME-aware `prefetchBodies` handle it. + +**Files:** +- Modify: `Sources/IMAPClient/IMAPClient.swift` +- Modify: `Sources/IMAPClient/FetchedEnvelope.swift` +- Modify: `Sources/SyncEngine/SyncCoordinator.swift` + +- [ ] **Step 1: Remove BODY[TEXT] from fetchEnvelopes fetch attributes** + +In `IMAPClient.swift`, update `fetchEnvelopes` — remove the `.bodySection` attribute: + +Change: +```swift +let responses = try await runner.run(.uidFetch( + .set(set), + [ + .envelope, + .flags, + .uid, + .rfc822Size, + .bodySection(peek: true, SectionSpecifier(kind: .text), nil), + ], + [] +)) +``` + +To: +```swift +let responses = try await runner.run(.uidFetch( + .set(set), + [ + .envelope, + .flags, + .uid, + .rfc822Size, + ], + [] +)) +``` + +- [ ] **Step 2: Simplify parseFetchResponses — remove bodyBuffer** + +In `IMAPClient.swift`, update `parseFetchResponses` to remove `bodyBuffer`. The body now comes from MIME-aware `prefetchBodies`, not from envelope fetch. + +Replace the method: + +```swift +private func parseFetchResponses(_ responses: [Response]) -> [FetchedEnvelope] { + var envelopes: [FetchedEnvelope] = [] + var currentUID: Int? + var currentEnvelope: Envelope? + var currentFlags: [Flag] = [] + var currentSize: Int = 0 + + for response in responses { + switch response { + case .fetch(let fetchResponse): + switch fetchResponse { + case .start, .startUID: + currentUID = nil + currentEnvelope = nil + currentFlags = [] + currentSize = 0 + case .simpleAttribute(let attr): + switch attr { + case .uid(let uid): + currentUID = Int(uid.rawValue) + case .envelope(let env): + currentEnvelope = env + case .flags(let flags): + currentFlags = flags + case .rfc822Size(let size): + currentSize = size + default: + break + } + case .streamingBegin, .streamingBytes, .streamingEnd: + break + case .finish: + if let uid = currentUID { + let envelope = buildFetchedEnvelope( + uid: uid, + envelope: currentEnvelope, + flags: currentFlags, + size: currentSize + ) + envelopes.append(envelope) + } + } + default: + break + } + } + + return envelopes +} +``` + +- [ ] **Step 3: Simplify buildFetchedEnvelope — remove bodyBuffer parameter** + +Replace the method (bodies come from `prefetchBodies` now): + +```swift +private func buildFetchedEnvelope( + uid: Int, + envelope: Envelope?, + flags: [Flag], + size: Int +) -> FetchedEnvelope { + let subject = envelope?.subject.flatMap { String(buffer: $0) } + let date = envelope?.date.map { String($0) } ?? "" + let messageId = envelope?.messageID.map { String($0) } + let inReplyTo = envelope?.inReplyTo.map { String($0) } + let from = envelope?.from.compactMap { extractEmailAddress($0) }.first + let to = envelope?.to.compactMap { extractEmailAddress($0) } ?? [] + let cc = envelope?.cc.compactMap { extractEmailAddress($0) } ?? [] + + return FetchedEnvelope( + uid: uid, + messageId: messageId, + inReplyTo: inReplyTo, + references: nil, + subject: subject, + from: from, + to: to, + cc: cc, + date: date, + snippet: nil, + bodyText: nil, + bodyHtml: nil, + isRead: flags.contains(.seen), + isFlagged: flags.contains(.flagged), + size: size + ) +} +``` + +- [ ] **Step 4: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 | tail -3 +``` + +Expected: All tests pass. Existing sync tests use MockIMAPClient which doesn't change behavior. The `fullMessages` mock provides MIME bodies during prefetch. + +- [ ] **Step 5: Commit** + +```bash +git add Sources/IMAPClient/IMAPClient.swift +git commit -m "remove inline body fetch from fetchEnvelopes, bodies now come from MIME-aware prefetchBodies" +``` + +--- + +### Task 14: Attachment download service + +**Files:** +- Modify: `Sources/SyncEngine/SyncCoordinator.swift` + +The spec defines an attachment download flow: check cachePath → if cached open QuickLook → if not fetch via `fetchSection` → save to disk → update `cachePath`. This orchestration lives in SyncCoordinator since it coordinates IMAP + MailStore. + +- [ ] **Step 1: Add downloadAttachment method** + +Add to `SyncCoordinator.swift`: + +```swift +// MARK: - Attachment Download + +/// Download an attachment on demand, cache it to disk, update the DB record. +/// Returns the local file URL for preview. +public func downloadAttachment( + attachmentId: String, + messageUid: Int, + mailboxName: String +) async throws -> URL { + // Look up the attachment record + guard let record = try store.attachment(id: attachmentId) else { + throw AttachmentError.notFound + } + + // Check if already cached + if let cachePath = record.cachePath { + let url = URL(fileURLWithPath: cachePath) + if FileManager.default.fileExists(atPath: cachePath) { + return url + } + } + + guard let sectionPath = record.sectionPath else { + throw AttachmentError.noSectionPath + } + + // Fetch the section data from IMAP + try await imapClient.connect() + let data = try await imapClient.fetchSection(uid: messageUid, mailbox: mailboxName, section: sectionPath) + try? await imapClient.disconnect() + + // Build cache directory + let cacheDir = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first! + .appendingPathComponent("MagnumOpus") + .appendingPathComponent(accountConfig.id) + .appendingPathComponent("attachments") + try FileManager.default.createDirectory(at: cacheDir, withIntermediateDirectories: true) + + // Determine file extension from filename + let ext = record.filename.flatMap { URL(fileURLWithPath: $0).pathExtension } ?? "bin" + let fileName = "\(attachmentId).\(ext)" + let fileURL = cacheDir.appendingPathComponent(fileName) + + // Write data to disk + try data.write(to: fileURL) + + // Update cache path in DB + try store.updateAttachmentCachePath(id: attachmentId, cachePath: fileURL.path) + + return fileURL +} + +public enum AttachmentError: Error { + case notFound + case noSectionPath +} +``` + +- [ ] **Step 2: Add `attachment(id:)` query to MailStore** + +Add to `MailStore.swift` in the `// MARK: - Attachments` section: + +```swift +public func attachment(id: String) throws -> AttachmentRecord? { + try dbWriter.read { db in + try AttachmentRecord.fetchOne(db, key: id) + } +} +``` + +- [ ] **Step 3: Build and run tests** + +```bash +cd Packages/MagnumOpusCore +swift build && swift test 2>&1 | tail -3 +``` + +Expected: Build and tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add Sources/SyncEngine/SyncCoordinator.swift Sources/MailStore/MailStore.swift +git commit -m "add attachment download service: fetch section from IMAP, cache to disk" +``` + +--- + +### Task 15: 25 MB per-file attachment size guard + +**Files:** +- Modify: `Sources/SMTPClient/MessageFormatter.swift` +- Modify: `Tests/SMTPClientTests/MessageFormatterTests.swift` + +- [ ] **Step 1: Write failing test** + +Add to `MessageFormatterTests.swift`: + +```swift +@Test("formatMultipart throws for file exceeding 25 MB") +func attachmentSizeGuard() { + let message = OutgoingMessage( + from: EmailAddress(address: "alice@example.com"), + to: [EmailAddress(address: "bob@example.com")], + subject: "Big file", + bodyText: "See attached.", + messageId: "test@example.com" + ) + // 26 MB attachment + let bigData = Data(repeating: 0xFF, count: 26 * 1024 * 1024) + let attachments: [(filename: String, mimeType: String, data: Data)] = [ + (filename: "huge.bin", mimeType: "application/octet-stream", data: bigData), + ] + + #expect(throws: MessageFormatterError.self) { + try MessageFormatter.formatMultipart(message, attachments: attachments) + } +} +``` + +- [ ] **Step 2: Add size guard and error type** + +In `MessageFormatter.swift`, add at the bottom of the file: + +```swift +public enum MessageFormatterError: Error { + case attachmentTooLarge(filename: String, size: Int) +} +``` + +Update `formatMultipart` signature to `throws` and add a size check at the start: + +```swift +public static func formatMultipart( + _ message: OutgoingMessage, + attachments: [(filename: String, mimeType: String, data: Data)] +) throws -> String { + let maxSize = 25 * 1024 * 1024 // 25 MB + for attachment in attachments { + if attachment.data.count > maxSize { + throw MessageFormatterError.attachmentTooLarge( + filename: attachment.filename, + size: attachment.data.count + ) + } + } + // ... rest of existing implementation unchanged +``` + +- [ ] **Step 3: Update non-throwing test calls to use `try`** + +The existing multipart tests now need `try` since `formatMultipart` throws. Update test calls from: +```swift +let formatted = MessageFormatter.formatMultipart(message, attachments: attachments) +``` +To: +```swift +let formatted = try MessageFormatter.formatMultipart(message, attachments: attachments) +``` + +- [ ] **Step 4: Run tests** + +```bash +cd Packages/MagnumOpusCore +swift test --filter MessageFormatterTests 2>&1 | tail -5 +``` + +Expected: All tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add Sources/SMTPClient/MessageFormatter.swift Tests/SMTPClientTests/MessageFormatterTests.swift +git commit -m "add 25 MB per-file attachment size guard to MessageFormatter" +``` + +--- + +### Task 16: Update periodic sync interval & add SyncEngine dependency on MIMEParser to Package.swift project.yml + +**Files:** +- Modify: `Sources/SyncEngine/SyncCoordinator.swift` +- Modify: `Apps/project.yml` + +- [ ] **Step 1: Update default periodic sync interval** + +In `SyncCoordinator.swift`, change the `startPeriodicSync` default interval: + +```swift +public func startPeriodicSync(interval: Duration = .seconds(900)) { +``` + +(Changed from 300s/5min to 900s/15min — IDLE is the fast path now.) + +- [ ] **Step 2: Add MIMEParser to project.yml** + +In `Apps/project.yml`, add MIMEParser to the dependencies for both macOS and iOS targets. Look at how other products are listed and add: + +```yaml +- package: MagnumOpusCore + product: MIMEParser +``` + +- [ ] **Step 3: Build and verify** + +```bash +cd Packages/MagnumOpusCore +swift build +``` + +- [ ] **Step 4: Commit** + +```bash +git add Sources/SyncEngine/SyncCoordinator.swift Apps/project.yml +git commit -m "bump periodic sync to 15min (IDLE is fast path), add MIMEParser to app targets" +``` + +--- + +### Task 17: Final verification + +- [ ] **Step 1: Run full test suite** + +```bash +cd Packages/MagnumOpusCore +swift test 2>&1 +``` + +Expected: All tests pass (original 111 + new tests). + +- [ ] **Step 2: Build Xcode project** + +```bash +cd /Users/felixfoertsch/Developer/MagnumOpus +xcodegen generate --spec Apps/project.yml --project Apps/ +``` + +Then: + +```bash +cd Apps +xcodebuild build -project MagnumOpus.xcodeproj -scheme MagnumOpus-macOS -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO 2>&1 | tail -10 +``` + +Expected: Build succeeds. + +- [ ] **Step 3: Commit any remaining changes** + +```bash +cd /Users/felixfoertsch/Developer/MagnumOpus +git status +``` + +If clean, proceed. Otherwise commit remaining changes. + +- [ ] **Step 4: Bump CalVer version** + +Update the CalVer version in `Apps/project.yml` to today's date if not already done: + +```yaml +MARKETING_VERSION: "2026.03.14" +``` + +(Already set from v0.4 — if this runs on a different day, update accordingly.) + +- [ ] **Step 5: Final commit** + +```bash +git add -A +git commit -m "v0.5: attachments (MIME parse, multipart send) + IMAP IDLE real-time mail" +```