2903 lines
81 KiB
Markdown
2903 lines
81 KiB
Markdown
# v0.5 Attachments & IMAP IDLE — Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Complete the email client with full attachment support (send and receive) and real-time email arrival via IMAP IDLE.
|
|
|
|
**Architecture:** Three subsystems: (1) MIMEParser module for parsing multipart MIME messages, (2) attachment sending via multipart/mixed MessageFormatter extension, (3) IMAPIdleClient actor for persistent INBOX monitoring. MIMEParser is a new module with no external dependencies. IDLE uses a separate NIO connection from the sync connection.
|
|
|
|
**Tech Stack:** Swift 6, swift-nio, swift-nio-imap, swift-nio-ssl, GRDB.swift, Swift Testing
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
### New Files
|
|
| File | Responsibility |
|
|
|------|---------------|
|
|
| `Sources/MIMEParser/MIMETypes.swift` | Public types: MIMEMessage, MIMEPart, MIMEAttachment, TransferEncoding, ContentDisposition |
|
|
| `Sources/MIMEParser/MIMEParser.swift` | Static parse/decode/boundary functions |
|
|
| `Sources/MIMEParser/RFC2047Decoder.swift` | Decode `=?charset?encoding?text?=` encoded words in filenames |
|
|
| `Sources/IMAPClient/IMAPIdleClient.swift` | Actor managing dedicated IDLE connection |
|
|
| `Sources/IMAPClient/IMAPIdleHandler.swift` | NIO ChannelInboundHandler for IDLE responses, delivers AsyncStream<IMAPIdleEvent> |
|
|
| `Tests/MIMEParserTests/MIMEParserTests.swift` | MIME parsing tests |
|
|
| `Tests/MIMEParserTests/RFC2047DecoderTests.swift` | RFC 2047 filename decoding tests |
|
|
| `Tests/IMAPClientTests/IMAPIdleClientTests.swift` | IDLE client tests (using mock handler) |
|
|
|
|
### Modified Files
|
|
| File | Changes |
|
|
|------|---------|
|
|
| `Package.swift` | Add MIMEParser target + test target |
|
|
| `Sources/MailStore/DatabaseSetup.swift` | Add v4_attachment migration |
|
|
| `Sources/MailStore/Records/AttachmentRecord.swift` | Add sectionPath column |
|
|
| `Sources/MailStore/Records/MessageRecord.swift` | Add hasAttachments column |
|
|
| `Sources/MailStore/MailStore.swift` | Add attachment CRUD, storeBodyWithAttachments, attachment download query |
|
|
| `Sources/MailStore/Queries.swift` | Read hasAttachments in toMessageSummary |
|
|
| `Sources/IMAPClient/IMAPClientProtocol.swift` | Add fetchFullMessage, fetchSection |
|
|
| `Sources/IMAPClient/IMAPClient.swift` | Implement fetchFullMessage, fetchSection; remove body from fetchEnvelopes |
|
|
| `Sources/SMTPClient/MessageFormatter.swift` | Add formatMultipart, base64Encode, 25MB guard |
|
|
| `Sources/SyncEngine/SyncCoordinator.swift` | MIME-aware prefetchBodies, IDLE start/stop, attachment download, credentials |
|
|
| `Tests/SyncEngineTests/MockIMAPClient.swift` | Add fetchFullMessage, fetchSection mocks |
|
|
| `Tests/SMTPClientTests/MessageFormatterTests.swift` | Add multipart formatting + size guard tests |
|
|
| `Tests/SyncEngineTests/SyncCoordinatorTests.swift` | Add MIME sync + IDLE integration tests |
|
|
| `Tests/MailStoreTests/MigrationTests.swift` | Add v4_attachment migration test |
|
|
|
|
### Deferred to App Layer
|
|
The spec's UI changes (ComposeViewModel attachments, attachment strip in ThreadDetail, file picker, paperclip icon in message list, inline image cid: replacement) are app-layer SwiftUI code outside MagnumOpusCore. They depend on the core infrastructure built in this plan. They will be wired up when integrating v0.5 into the app targets — same pattern as v0.3/v0.4 where core logic ships first, then UI wraps it.
|
|
|
|
---
|
|
|
|
## Chunk 1: MIMEParser Module — Types & Basic Parsing
|
|
|
|
### Task 1: Add MIMEParser target to Package.swift
|
|
|
|
**Files:**
|
|
- Modify: `Package.swift`
|
|
|
|
- [ ] **Step 1: Add MIMEParser target and test target**
|
|
|
|
In `Package.swift`, add:
|
|
- A product: `.library(name: "MIMEParser", targets: ["MIMEParser"])`
|
|
- A target: `.target(name: "MIMEParser")` (no dependencies — pure Swift)
|
|
- A test target: `.testTarget(name: "MIMEParserTests", dependencies: ["MIMEParser"])`
|
|
|
|
Add after the existing `SMTPClient` product/target entries.
|
|
|
|
- [ ] **Step 2: Create source directory**
|
|
|
|
```bash
|
|
mkdir -p Packages/MagnumOpusCore/Sources/MIMEParser
|
|
mkdir -p Packages/MagnumOpusCore/Tests/MIMEParserTests
|
|
```
|
|
|
|
- [ ] **Step 3: Verify build**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build
|
|
```
|
|
|
|
Expected: Build succeeds (empty target is fine with directory existing).
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Package.swift Sources/MIMEParser Tests/MIMEParserTests
|
|
git commit -m "add MIMEParser module target to Package.swift"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: MIME types
|
|
|
|
**Files:**
|
|
- Create: `Sources/MIMEParser/MIMETypes.swift`
|
|
|
|
- [ ] **Step 1: Write MIMETypes.swift**
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
public enum TransferEncoding: String, Sendable {
|
|
case base64
|
|
case quotedPrintable = "quoted-printable"
|
|
case sevenBit = "7bit"
|
|
case eightBit = "8bit"
|
|
case binary
|
|
}
|
|
|
|
public enum ContentDisposition: Sendable {
|
|
case inline
|
|
case attachment
|
|
}
|
|
|
|
public struct MIMEPart: Sendable {
|
|
public var headers: [String: String]
|
|
public var contentType: String
|
|
public var charset: String?
|
|
public var transferEncoding: TransferEncoding
|
|
public var disposition: ContentDisposition?
|
|
public var filename: String?
|
|
public var contentId: String?
|
|
public var body: Data
|
|
public var subparts: [MIMEPart]
|
|
|
|
public init(
|
|
headers: [String: String] = [:],
|
|
contentType: String = "text/plain",
|
|
charset: String? = "utf-8",
|
|
transferEncoding: TransferEncoding = .sevenBit,
|
|
disposition: ContentDisposition? = nil,
|
|
filename: String? = nil,
|
|
contentId: String? = nil,
|
|
body: Data = Data(),
|
|
subparts: [MIMEPart] = []
|
|
) {
|
|
self.headers = headers
|
|
self.contentType = contentType
|
|
self.charset = charset
|
|
self.transferEncoding = transferEncoding
|
|
self.disposition = disposition
|
|
self.filename = filename
|
|
self.contentId = contentId
|
|
self.body = body
|
|
self.subparts = subparts
|
|
}
|
|
}
|
|
|
|
public struct MIMEAttachment: Sendable {
|
|
public var filename: String
|
|
public var mimeType: String
|
|
public var size: Int
|
|
public var contentId: String?
|
|
public var sectionPath: String
|
|
public var isInline: Bool
|
|
|
|
public init(
|
|
filename: String,
|
|
mimeType: String,
|
|
size: Int,
|
|
contentId: String? = nil,
|
|
sectionPath: String,
|
|
isInline: Bool = false
|
|
) {
|
|
self.filename = filename
|
|
self.mimeType = mimeType
|
|
self.size = size
|
|
self.contentId = contentId
|
|
self.sectionPath = sectionPath
|
|
self.isInline = isInline
|
|
}
|
|
}
|
|
|
|
public struct MIMEMessage: Sendable {
|
|
public var headers: [String: String]
|
|
public var parts: [MIMEPart]
|
|
public var textBody: String?
|
|
public var htmlBody: String?
|
|
public var attachments: [MIMEAttachment]
|
|
public var inlineImages: [MIMEAttachment]
|
|
|
|
public init(
|
|
headers: [String: String] = [:],
|
|
parts: [MIMEPart] = [],
|
|
textBody: String? = nil,
|
|
htmlBody: String? = nil,
|
|
attachments: [MIMEAttachment] = [],
|
|
inlineImages: [MIMEAttachment] = []
|
|
) {
|
|
self.headers = headers
|
|
self.parts = parts
|
|
self.textBody = textBody
|
|
self.htmlBody = htmlBody
|
|
self.attachments = attachments
|
|
self.inlineImages = inlineImages
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Verify build**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add Sources/MIMEParser/MIMETypes.swift
|
|
git commit -m "add MIME types: MIMEMessage, MIMEPart, MIMEAttachment, TransferEncoding"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: RFC 2047 decoder
|
|
|
|
**Files:**
|
|
- Create: `Sources/MIMEParser/RFC2047Decoder.swift`
|
|
- Create: `Tests/MIMEParserTests/RFC2047DecoderTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing tests for RFC 2047 decoding**
|
|
|
|
```swift
|
|
import Testing
|
|
import Foundation
|
|
@testable import MIMEParser
|
|
|
|
@Suite("RFC2047Decoder")
|
|
struct RFC2047DecoderTests {
|
|
|
|
@Test("plain ASCII filename passes through unchanged")
|
|
func plainAscii() {
|
|
let result = RFC2047Decoder.decode("report.pdf")
|
|
#expect(result == "report.pdf")
|
|
}
|
|
|
|
@Test("base64 encoded UTF-8 filename decoded correctly")
|
|
func base64Utf8() {
|
|
// "Bericht.pdf" in base64
|
|
let encoded = "=?utf-8?B?QmVyaWNodC5wZGY=?="
|
|
let result = RFC2047Decoder.decode(encoded)
|
|
#expect(result == "Bericht.pdf")
|
|
}
|
|
|
|
@Test("quoted-printable encoded UTF-8 filename decoded correctly")
|
|
func quotedPrintableUtf8() {
|
|
// "Grüße.txt" — ü = =C3=BC, ß = =C3=9F
|
|
let encoded = "=?utf-8?Q?Gr=C3=BC=C3=9Fe.txt?="
|
|
let result = RFC2047Decoder.decode(encoded)
|
|
#expect(result == "Grüße.txt")
|
|
}
|
|
|
|
@Test("multiple encoded words concatenated")
|
|
func multipleEncodedWords() {
|
|
let encoded = "=?utf-8?B?SGVsbG8=?= =?utf-8?B?V29ybGQ=?="
|
|
let result = RFC2047Decoder.decode(encoded)
|
|
#expect(result == "HelloWorld")
|
|
}
|
|
|
|
@Test("ISO-8859-1 encoded filename decoded correctly")
|
|
func iso88591() {
|
|
// "café" — é = 0xE9 in ISO-8859-1, base64 of "café" in ISO-8859-1 is "Y2Fm6Q=="
|
|
let encoded = "=?iso-8859-1?B?Y2Fm6Q==?="
|
|
let result = RFC2047Decoder.decode(encoded)
|
|
#expect(result == "café")
|
|
}
|
|
|
|
@Test("underscores in Q-encoding replaced with spaces")
|
|
func qEncodingUnderscores() {
|
|
let encoded = "=?utf-8?Q?my_file_name.pdf?="
|
|
let result = RFC2047Decoder.decode(encoded)
|
|
#expect(result == "my file name.pdf")
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter RFC2047DecoderTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: Compilation error — `RFC2047Decoder` not defined.
|
|
|
|
- [ ] **Step 3: Implement RFC2047Decoder**
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
public enum RFC2047Decoder {
|
|
/// Decode RFC 2047 encoded words in a string.
|
|
/// Pattern: =?charset?encoding?encoded_text?=
|
|
public static func decode(_ input: String) -> String {
|
|
let pattern = #"=\?([^?]+)\?([BbQq])\?([^?]*)\?="#
|
|
guard let regex = try? NSRegularExpression(pattern: pattern) else {
|
|
return input
|
|
}
|
|
|
|
let nsInput = input as NSString
|
|
let matches = regex.matches(in: input, range: NSRange(location: 0, length: nsInput.length))
|
|
|
|
guard !matches.isEmpty else { return input }
|
|
|
|
var result = ""
|
|
var lastEnd = 0
|
|
|
|
for match in matches {
|
|
let matchRange = match.range
|
|
// Add any non-encoded text between matches (skip whitespace between adjacent encoded words)
|
|
let gap = nsInput.substring(with: NSRange(location: lastEnd, length: matchRange.location - lastEnd))
|
|
let trimmedGap = gap.trimmingCharacters(in: .whitespaces)
|
|
if !trimmedGap.isEmpty || lastEnd == 0 {
|
|
// Only add gap if it's not just whitespace between encoded words
|
|
if lastEnd == 0 && matchRange.location > 0 {
|
|
result += gap
|
|
} else if !trimmedGap.isEmpty {
|
|
result += gap
|
|
}
|
|
}
|
|
|
|
let charset = nsInput.substring(with: match.range(at: 1))
|
|
let encoding = nsInput.substring(with: match.range(at: 2)).uppercased()
|
|
let encodedText = nsInput.substring(with: match.range(at: 3))
|
|
|
|
let cfEncoding = CFStringConvertIANACharSetNameToEncoding(charset as CFString)
|
|
let nsEncoding = CFStringConvertEncodingToNSStringEncoding(cfEncoding)
|
|
|
|
let decoded: String?
|
|
if encoding == "B" {
|
|
guard let data = Data(base64Encoded: encodedText) else {
|
|
result += nsInput.substring(with: matchRange)
|
|
lastEnd = matchRange.location + matchRange.length
|
|
continue
|
|
}
|
|
decoded = String(data: data, encoding: String.Encoding(rawValue: nsEncoding))
|
|
} else {
|
|
// Q encoding: like quoted-printable but underscores represent spaces
|
|
let withSpaces = encodedText.replacingOccurrences(of: "_", with: " ")
|
|
let data = decodeQuotedPrintableBytes(withSpaces)
|
|
decoded = String(data: data, encoding: String.Encoding(rawValue: nsEncoding))
|
|
}
|
|
|
|
result += decoded ?? nsInput.substring(with: matchRange)
|
|
lastEnd = matchRange.location + matchRange.length
|
|
}
|
|
|
|
// Append any trailing non-encoded text
|
|
if lastEnd < nsInput.length {
|
|
result += nsInput.substring(from: lastEnd)
|
|
}
|
|
|
|
return result
|
|
}
|
|
|
|
private static func decodeQuotedPrintableBytes(_ input: String) -> Data {
|
|
var data = Data()
|
|
var i = input.startIndex
|
|
while i < input.endIndex {
|
|
if input[i] == "=" {
|
|
let hexStart = input.index(after: i)
|
|
guard hexStart < input.endIndex else {
|
|
data.append(contentsOf: "=".utf8)
|
|
break
|
|
}
|
|
let hexEnd = input.index(hexStart, offsetBy: 1, limitedBy: input.endIndex) ?? input.endIndex
|
|
guard hexEnd < input.endIndex else {
|
|
data.append(contentsOf: String(input[i...]).utf8)
|
|
break
|
|
}
|
|
let nextAfterHex = input.index(after: hexEnd)
|
|
let hex = String(input[hexStart...hexEnd])
|
|
if let byte = UInt8(hex, radix: 16) {
|
|
data.append(byte)
|
|
i = nextAfterHex
|
|
} else {
|
|
data.append(contentsOf: "=".utf8)
|
|
i = hexStart
|
|
}
|
|
} else {
|
|
data.append(contentsOf: String(input[i]).utf8)
|
|
i = input.index(after: i)
|
|
}
|
|
}
|
|
return data
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter RFC2047DecoderTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: All 6 tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Sources/MIMEParser/RFC2047Decoder.swift Tests/MIMEParserTests/RFC2047DecoderTests.swift
|
|
git commit -m "add RFC 2047 encoded word decoder for MIME filenames"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: MIMEParser core — header parsing, content decoding, boundary generation
|
|
|
|
**Files:**
|
|
- Create: `Sources/MIMEParser/MIMEParser.swift`
|
|
- Create: `Tests/MIMEParserTests/MIMEParserTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing tests for content decoding and boundary generation**
|
|
|
|
```swift
|
|
import Testing
|
|
import Foundation
|
|
@testable import MIMEParser
|
|
|
|
@Suite("MIMEParser")
|
|
struct MIMEParserTests {
|
|
|
|
// MARK: - Content Decoding
|
|
|
|
@Test("decode base64 content")
|
|
func decodeBase64() {
|
|
let encoded = "SGVsbG8gV29ybGQ="
|
|
let data = MIMEParser.decodeContent(encoded, encoding: .base64)
|
|
#expect(String(data: data, encoding: .utf8) == "Hello World")
|
|
}
|
|
|
|
@Test("decode quoted-printable content")
|
|
func decodeQuotedPrintable() {
|
|
let encoded = "Gr=C3=BC=C3=9Fe"
|
|
let data = MIMEParser.decodeContent(encoded, encoding: .quotedPrintable)
|
|
#expect(String(data: data, encoding: .utf8) == "Grüße")
|
|
}
|
|
|
|
@Test("decode 7bit content passes through")
|
|
func decode7bit() {
|
|
let text = "Hello World"
|
|
let data = MIMEParser.decodeContent(text, encoding: .sevenBit)
|
|
#expect(String(data: data, encoding: .utf8) == "Hello World")
|
|
}
|
|
|
|
@Test("boundary generation produces unique strings with =_ prefix")
|
|
func boundaryGeneration() {
|
|
let b1 = MIMEParser.generateBoundary()
|
|
let b2 = MIMEParser.generateBoundary()
|
|
#expect(b1 != b2)
|
|
#expect(b1.hasPrefix("=_MagnumOpus_"))
|
|
#expect(b2.hasPrefix("=_MagnumOpus_"))
|
|
}
|
|
|
|
// MARK: - Single-part Parsing
|
|
|
|
@Test("parse single-part text/plain message")
|
|
func parseSinglePartText() {
|
|
let raw = """
|
|
Content-Type: text/plain; charset=utf-8\r
|
|
Content-Transfer-Encoding: 7bit\r
|
|
\r
|
|
Hello, this is the body.
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.textBody == "Hello, this is the body.")
|
|
#expect(message.htmlBody == nil)
|
|
#expect(message.attachments.isEmpty)
|
|
}
|
|
|
|
// MARK: - Multipart Parsing
|
|
|
|
@Test("parse multipart/mixed with text and one attachment")
|
|
func parseMultipartMixed() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="----boundary123"\r
|
|
\r
|
|
------boundary123\r
|
|
Content-Type: text/plain; charset=utf-8\r
|
|
Content-Transfer-Encoding: 7bit\r
|
|
\r
|
|
Hello from the body.\r
|
|
------boundary123\r
|
|
Content-Type: application/pdf; name="report.pdf"\r
|
|
Content-Disposition: attachment; filename="report.pdf"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
SGVsbG8=\r
|
|
------boundary123--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.textBody == "Hello from the body.")
|
|
#expect(message.attachments.count == 1)
|
|
#expect(message.attachments.first?.filename == "report.pdf")
|
|
#expect(message.attachments.first?.mimeType == "application/pdf")
|
|
#expect(message.attachments.first?.sectionPath == "2")
|
|
#expect(message.attachments.first?.isInline == false)
|
|
}
|
|
|
|
@Test("parse multipart/alternative extracts text and html bodies")
|
|
func parseMultipartAlternative() {
|
|
let raw = """
|
|
Content-Type: multipart/alternative; boundary="alt-boundary"\r
|
|
\r
|
|
--alt-boundary\r
|
|
Content-Type: text/plain; charset=utf-8\r
|
|
\r
|
|
Plain text body\r
|
|
--alt-boundary\r
|
|
Content-Type: text/html; charset=utf-8\r
|
|
\r
|
|
<p>HTML body</p>\r
|
|
--alt-boundary--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.textBody == "Plain text body")
|
|
#expect(message.htmlBody == "<p>HTML body</p>")
|
|
#expect(message.attachments.isEmpty)
|
|
}
|
|
|
|
@Test("parse multipart/related with inline image")
|
|
func parseMultipartRelated() {
|
|
let raw = """
|
|
Content-Type: multipart/related; boundary="rel-boundary"\r
|
|
\r
|
|
--rel-boundary\r
|
|
Content-Type: text/html; charset=utf-8\r
|
|
\r
|
|
<p>Image: <img src="cid:img001"></p>\r
|
|
--rel-boundary\r
|
|
Content-Type: image/png\r
|
|
Content-ID: <img001>\r
|
|
Content-Disposition: inline\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
iVBORw0KGgo=\r
|
|
--rel-boundary--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.htmlBody == "<p>Image: <img src=\"cid:img001\"></p>")
|
|
#expect(message.inlineImages.count == 1)
|
|
#expect(message.inlineImages.first?.contentId == "img001")
|
|
#expect(message.inlineImages.first?.isInline == true)
|
|
}
|
|
|
|
@Test("parse nested multipart/mixed containing multipart/alternative")
|
|
func parseNestedMultipart() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="outer"\r
|
|
\r
|
|
--outer\r
|
|
Content-Type: multipart/alternative; boundary="inner"\r
|
|
\r
|
|
--inner\r
|
|
Content-Type: text/plain\r
|
|
\r
|
|
Plain text\r
|
|
--inner\r
|
|
Content-Type: text/html\r
|
|
\r
|
|
<p>HTML</p>\r
|
|
--inner--\r
|
|
--outer\r
|
|
Content-Type: application/pdf; name="doc.pdf"\r
|
|
Content-Disposition: attachment; filename="doc.pdf"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
AAAA\r
|
|
--outer--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.textBody == "Plain text")
|
|
#expect(message.htmlBody == "<p>HTML</p>")
|
|
#expect(message.attachments.count == 1)
|
|
#expect(message.attachments.first?.filename == "doc.pdf")
|
|
}
|
|
|
|
@Test("section paths assigned correctly for nested parts")
|
|
func sectionPaths() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="outer"\r
|
|
\r
|
|
--outer\r
|
|
Content-Type: text/plain\r
|
|
\r
|
|
Body text\r
|
|
--outer\r
|
|
Content-Type: application/pdf; name="a.pdf"\r
|
|
Content-Disposition: attachment; filename="a.pdf"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
AAAA\r
|
|
--outer\r
|
|
Content-Type: image/jpeg; name="b.jpg"\r
|
|
Content-Disposition: attachment; filename="b.jpg"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
BBBB\r
|
|
--outer--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.attachments.count == 2)
|
|
#expect(message.attachments[0].sectionPath == "2")
|
|
#expect(message.attachments[1].sectionPath == "3")
|
|
}
|
|
|
|
@Test("extract filename from Content-Type name parameter when no Content-Disposition")
|
|
func filenameFromContentType() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="bound"\r
|
|
\r
|
|
--bound\r
|
|
Content-Type: text/plain\r
|
|
\r
|
|
Body\r
|
|
--bound\r
|
|
Content-Type: application/octet-stream; name="data.bin"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
AAAA\r
|
|
--bound--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.attachments.count == 1)
|
|
#expect(message.attachments.first?.filename == "data.bin")
|
|
}
|
|
|
|
@Test("estimate decoded size from base64 content")
|
|
func base64SizeEstimate() {
|
|
// 8 base64 chars = 6 decoded bytes
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="bound"\r
|
|
\r
|
|
--bound\r
|
|
Content-Type: text/plain\r
|
|
\r
|
|
Body\r
|
|
--bound\r
|
|
Content-Type: application/pdf; name="f.pdf"\r
|
|
Content-Disposition: attachment; filename="f.pdf"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
AAAAAAAA\r
|
|
--bound--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.attachments.first?.size == 6)
|
|
}
|
|
|
|
@Test("handle malformed MIME gracefully — missing boundary")
|
|
func malformedMissingBoundary() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed\r
|
|
\r
|
|
Some text without proper boundary markers.
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
// Should not crash; treat as single-part
|
|
#expect(message.attachments.isEmpty)
|
|
}
|
|
|
|
@Test("RFC 2047 encoded filename decoded")
|
|
func rfc2047Filename() {
|
|
let raw = """
|
|
Content-Type: multipart/mixed; boundary="bound"\r
|
|
\r
|
|
--bound\r
|
|
Content-Type: text/plain\r
|
|
\r
|
|
Body\r
|
|
--bound\r
|
|
Content-Type: application/pdf; name="=?utf-8?B?QmVyaWNodC5wZGY=?="\r
|
|
Content-Disposition: attachment; filename="=?utf-8?B?QmVyaWNodC5wZGY=?="\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
AAAA\r
|
|
--bound--
|
|
"""
|
|
let message = MIMEParser.parse(raw)
|
|
#expect(message.attachments.first?.filename == "Bericht.pdf")
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MIMEParserTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: Compilation error — `MIMEParser` not defined.
|
|
|
|
- [ ] **Step 3: Implement MIMEParser**
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
public enum MIMEParser {
|
|
|
|
// MARK: - Public API
|
|
|
|
/// Parse a raw MIME message into a structured tree of parts
|
|
public static func parse(_ rawMessage: String) -> MIMEMessage {
|
|
let (headers, body) = splitHeadersAndBody(rawMessage)
|
|
let contentType = headers["content-type"] ?? "text/plain"
|
|
|
|
if contentType.lowercased().contains("multipart/") {
|
|
guard let boundary = extractBoundary(contentType) else {
|
|
// Malformed: multipart without boundary — treat as plain text
|
|
return MIMEMessage(
|
|
headers: headers,
|
|
textBody: body.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
)
|
|
}
|
|
let parts = splitOnBoundary(body, boundary: boundary)
|
|
let parsedParts = parts.enumerated().map { (index, partString) in
|
|
parsePart(partString, sectionPrefix: "", index: index + 1)
|
|
}
|
|
|
|
var message = MIMEMessage(headers: headers, parts: parsedParts)
|
|
extractBodiesAndAttachments(from: parsedParts, contentType: contentType, into: &message, sectionPrefix: "")
|
|
return message
|
|
} else {
|
|
// Single-part message
|
|
let transferEncoding = parseTransferEncoding(headers["content-transfer-encoding"])
|
|
let decoded = decodeContent(body, encoding: transferEncoding)
|
|
|
|
if contentType.lowercased().contains("text/html") {
|
|
return MIMEMessage(headers: headers, htmlBody: String(data: decoded, encoding: .utf8))
|
|
} else {
|
|
return MIMEMessage(
|
|
headers: headers,
|
|
textBody: String(data: decoded, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
)
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Decode content based on Content-Transfer-Encoding
|
|
public static func decodeContent(_ content: String, encoding: TransferEncoding) -> Data {
|
|
switch encoding {
|
|
case .base64:
|
|
let cleaned = content.filter { !$0.isWhitespace }
|
|
return Data(base64Encoded: cleaned) ?? Data(content.utf8)
|
|
case .quotedPrintable:
|
|
return decodeQuotedPrintable(content)
|
|
case .sevenBit, .eightBit, .binary:
|
|
return Data(content.utf8)
|
|
}
|
|
}
|
|
|
|
/// Generate a unique MIME boundary string
|
|
public static func generateBoundary() -> String {
|
|
"=_MagnumOpus_\(UUID().uuidString)"
|
|
}
|
|
|
|
// MARK: - Header Parsing
|
|
|
|
private static func splitHeadersAndBody(_ raw: String) -> ([String: String], String) {
|
|
// Split on first blank line (CRLF CRLF or LF LF)
|
|
let separator: String
|
|
if raw.contains("\r\n\r\n") {
|
|
separator = "\r\n\r\n"
|
|
} else if raw.contains("\n\n") {
|
|
separator = "\n\n"
|
|
} else {
|
|
return ([:], raw)
|
|
}
|
|
|
|
guard let range = raw.range(of: separator) else {
|
|
return ([:], raw)
|
|
}
|
|
|
|
let headerSection = String(raw[raw.startIndex..<range.lowerBound])
|
|
let bodySection = String(raw[range.upperBound...])
|
|
return (parseHeaders(headerSection), bodySection)
|
|
}
|
|
|
|
private static func parseHeaders(_ section: String) -> [String: String] {
|
|
var headers: [String: String] = [:]
|
|
let lineBreak = section.contains("\r\n") ? "\r\n" : "\n"
|
|
let lines = section.components(separatedBy: lineBreak)
|
|
|
|
var currentKey: String?
|
|
var currentValue: String = ""
|
|
|
|
for line in lines {
|
|
if line.isEmpty { continue }
|
|
|
|
if line.first == " " || line.first == "\t" {
|
|
// Continuation of previous header (folded)
|
|
currentValue += " " + line.trimmingCharacters(in: .whitespaces)
|
|
} else if let colonIndex = line.firstIndex(of: ":") {
|
|
// Save previous header
|
|
if let key = currentKey {
|
|
headers[key.lowercased()] = currentValue
|
|
}
|
|
currentKey = String(line[..<colonIndex]).trimmingCharacters(in: .whitespaces)
|
|
currentValue = String(line[line.index(after: colonIndex)...]).trimmingCharacters(in: .whitespaces)
|
|
}
|
|
}
|
|
// Save last header
|
|
if let key = currentKey {
|
|
headers[key.lowercased()] = currentValue
|
|
}
|
|
|
|
return headers
|
|
}
|
|
|
|
// MARK: - Boundary / Part Splitting
|
|
|
|
private static func extractBoundary(_ contentType: String) -> String? {
|
|
// Look for boundary="value" or boundary=value
|
|
let lower = contentType.lowercased()
|
|
guard let boundaryRange = lower.range(of: "boundary=") else { return nil }
|
|
var value = String(contentType[boundaryRange.upperBound...])
|
|
|
|
// Strip leading quote
|
|
if value.hasPrefix("\"") {
|
|
value = String(value.dropFirst())
|
|
if let endQuote = value.firstIndex(of: "\"") {
|
|
value = String(value[..<endQuote])
|
|
}
|
|
} else {
|
|
// Unquoted — stop at semicolon or whitespace
|
|
if let end = value.firstIndex(where: { $0 == ";" || $0.isWhitespace }) {
|
|
value = String(value[..<end])
|
|
}
|
|
}
|
|
|
|
return value
|
|
}
|
|
|
|
private static func splitOnBoundary(_ body: String, boundary: String) -> [String] {
|
|
let delimiter = "--\(boundary)"
|
|
let terminator = "--\(boundary)--"
|
|
let lineBreak = body.contains("\r\n") ? "\r\n" : "\n"
|
|
|
|
var parts: [String] = []
|
|
let lines = body.components(separatedBy: lineBreak)
|
|
var currentPart: [String]? = nil
|
|
|
|
for line in lines {
|
|
let trimmed = line.trimmingCharacters(in: .whitespaces)
|
|
if trimmed == terminator || trimmed.hasPrefix(terminator) {
|
|
if let part = currentPart {
|
|
parts.append(part.joined(separator: lineBreak))
|
|
}
|
|
break
|
|
} else if trimmed == delimiter || trimmed.hasPrefix(delimiter) {
|
|
if let part = currentPart {
|
|
parts.append(part.joined(separator: lineBreak))
|
|
}
|
|
currentPart = []
|
|
} else if currentPart != nil {
|
|
currentPart!.append(line)
|
|
}
|
|
}
|
|
|
|
return parts
|
|
}
|
|
|
|
// MARK: - Part Parsing
|
|
|
|
private static func parsePart(_ partString: String, sectionPrefix: String, index: Int) -> MIMEPart {
|
|
let (headers, body) = splitHeadersAndBody(partString)
|
|
let contentType = headers["content-type"] ?? "text/plain"
|
|
let transferEncoding = parseTransferEncoding(headers["content-transfer-encoding"])
|
|
let charset = extractParameter(contentType, name: "charset")
|
|
let disposition = parseDisposition(headers["content-disposition"])
|
|
let contentId = extractContentId(headers["content-id"])
|
|
|
|
var filename = extractParameter(headers["content-disposition"] ?? "", name: "filename")
|
|
if filename == nil {
|
|
filename = extractParameter(contentType, name: "name")
|
|
}
|
|
// Decode RFC 2047 encoded filenames
|
|
if let encoded = filename {
|
|
filename = RFC2047Decoder.decode(encoded)
|
|
}
|
|
|
|
let section = sectionPrefix.isEmpty ? "\(index)" : "\(sectionPrefix).\(index)"
|
|
|
|
// Check for nested multipart
|
|
if contentType.lowercased().contains("multipart/") {
|
|
if let boundary = extractBoundary(contentType) {
|
|
let subparts = splitOnBoundary(body, boundary: boundary)
|
|
let parsedSubparts = subparts.enumerated().map { (i, s) in
|
|
parsePart(s, sectionPrefix: section, index: i + 1)
|
|
}
|
|
return MIMEPart(
|
|
headers: headers,
|
|
contentType: contentType.components(separatedBy: ";").first?.trimmingCharacters(in: .whitespaces).lowercased() ?? contentType,
|
|
charset: charset,
|
|
transferEncoding: transferEncoding,
|
|
disposition: disposition,
|
|
filename: filename,
|
|
contentId: contentId,
|
|
body: Data(),
|
|
subparts: parsedSubparts
|
|
)
|
|
}
|
|
}
|
|
|
|
let decodedBody = decodeContent(body, encoding: transferEncoding)
|
|
let baseContentType = contentType.components(separatedBy: ";").first?.trimmingCharacters(in: .whitespaces).lowercased() ?? contentType
|
|
|
|
return MIMEPart(
|
|
headers: headers,
|
|
contentType: baseContentType,
|
|
charset: charset,
|
|
transferEncoding: transferEncoding,
|
|
disposition: disposition,
|
|
filename: filename,
|
|
contentId: contentId,
|
|
body: decodedBody,
|
|
subparts: []
|
|
)
|
|
}
|
|
|
|
// MARK: - Body & Attachment Extraction
|
|
|
|
private static func extractBodiesAndAttachments(
|
|
from parts: [MIMEPart],
|
|
contentType: String,
|
|
into message: inout MIMEMessage,
|
|
sectionPrefix: String
|
|
) {
|
|
let lowerType = contentType.lowercased()
|
|
|
|
if lowerType.contains("multipart/alternative") {
|
|
for part in parts {
|
|
if !part.subparts.isEmpty {
|
|
extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "")
|
|
} else if part.contentType == "text/plain" && message.textBody == nil {
|
|
message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
} else if part.contentType == "text/html" && message.htmlBody == nil {
|
|
message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
}
|
|
}
|
|
} else if lowerType.contains("multipart/related") {
|
|
// First part is the HTML body, rest are inline resources
|
|
for (index, part) in parts.enumerated() {
|
|
if index == 0 {
|
|
if !part.subparts.isEmpty {
|
|
extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "")
|
|
} else if part.contentType == "text/html" {
|
|
message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
} else if part.contentType == "text/plain" {
|
|
message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
}
|
|
} else {
|
|
let sectionIndex = sectionPrefix.isEmpty ? "\(index + 1)" : "\(sectionPrefix).\(index + 1)"
|
|
let attachment = MIMEAttachment(
|
|
filename: part.filename ?? "inline-\(index)",
|
|
mimeType: part.contentType,
|
|
size: estimateDecodedSize(part),
|
|
contentId: part.contentId,
|
|
sectionPath: sectionIndex,
|
|
isInline: true
|
|
)
|
|
message.inlineImages.append(attachment)
|
|
}
|
|
}
|
|
} else {
|
|
// multipart/mixed or unknown multipart
|
|
var bodyFound = false
|
|
for (index, part) in parts.enumerated() {
|
|
let sectionIndex = sectionPrefix.isEmpty ? "\(index + 1)" : "\(sectionPrefix).\(index + 1)"
|
|
|
|
if !part.subparts.isEmpty {
|
|
// Nested multipart — recurse
|
|
extractBodiesAndAttachments(from: part.subparts, contentType: part.contentType, into: &message, sectionPrefix: "")
|
|
bodyFound = true
|
|
} else if !bodyFound && part.disposition != .attachment && part.contentType.hasPrefix("text/") {
|
|
if part.contentType == "text/html" {
|
|
message.htmlBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
} else {
|
|
message.textBody = String(data: part.body, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
}
|
|
bodyFound = true
|
|
} else if part.disposition == .attachment || part.filename != nil || !part.contentType.hasPrefix("text/") {
|
|
let attachment = MIMEAttachment(
|
|
filename: part.filename ?? "attachment-\(index + 1)",
|
|
mimeType: part.contentType,
|
|
size: estimateDecodedSize(part),
|
|
contentId: part.contentId,
|
|
sectionPath: sectionIndex,
|
|
isInline: part.disposition == .inline
|
|
)
|
|
if part.disposition == .inline {
|
|
message.inlineImages.append(attachment)
|
|
} else {
|
|
message.attachments.append(attachment)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// MARK: - Helper Functions
|
|
|
|
private static func parseTransferEncoding(_ value: String?) -> TransferEncoding {
|
|
guard let value = value?.trimmingCharacters(in: .whitespaces).lowercased() else { return .sevenBit }
|
|
return TransferEncoding(rawValue: value) ?? .sevenBit
|
|
}
|
|
|
|
private static func parseDisposition(_ value: String?) -> ContentDisposition? {
|
|
guard let value = value?.lowercased() else { return nil }
|
|
if value.hasPrefix("inline") { return .inline }
|
|
if value.hasPrefix("attachment") { return .attachment }
|
|
return nil
|
|
}
|
|
|
|
private static func extractParameter(_ headerValue: String, name: String) -> String? {
|
|
let lower = headerValue.lowercased()
|
|
let search = "\(name.lowercased())="
|
|
guard let range = lower.range(of: search) else { return nil }
|
|
var value = String(headerValue[range.upperBound...])
|
|
if value.hasPrefix("\"") {
|
|
value = String(value.dropFirst())
|
|
if let endQuote = value.firstIndex(of: "\"") {
|
|
value = String(value[..<endQuote])
|
|
}
|
|
} else {
|
|
if let end = value.firstIndex(where: { $0 == ";" || $0.isWhitespace }) {
|
|
value = String(value[..<end])
|
|
}
|
|
}
|
|
return value.isEmpty ? nil : value
|
|
}
|
|
|
|
private static func extractContentId(_ value: String?) -> String? {
|
|
guard var cid = value?.trimmingCharacters(in: .whitespaces) else { return nil }
|
|
if cid.hasPrefix("<") { cid = String(cid.dropFirst()) }
|
|
if cid.hasSuffix(">") { cid = String(cid.dropLast()) }
|
|
return cid.isEmpty ? nil : cid
|
|
}
|
|
|
|
private static func estimateDecodedSize(_ part: MIMEPart) -> Int {
|
|
if part.transferEncoding == .base64 {
|
|
// base64 inflates ~33%: decoded = encoded * 3 / 4
|
|
let base64Length = part.body.count
|
|
// But body is already decoded at this point, so use body.count directly
|
|
return part.body.count
|
|
}
|
|
return part.body.count
|
|
}
|
|
|
|
private static func decodeQuotedPrintable(_ input: String) -> Data {
|
|
var data = Data()
|
|
let lines = input.components(separatedBy: "\n")
|
|
|
|
for (lineIndex, line) in lines.enumerated() {
|
|
var processedLine = line
|
|
if processedLine.hasSuffix("\r") {
|
|
processedLine = String(processedLine.dropLast())
|
|
}
|
|
|
|
// Check for soft line break
|
|
if processedLine.hasSuffix("=") {
|
|
processedLine = String(processedLine.dropLast())
|
|
data.append(contentsOf: decodeQPLine(processedLine))
|
|
} else {
|
|
data.append(contentsOf: decodeQPLine(processedLine))
|
|
if lineIndex < lines.count - 1 {
|
|
data.append(contentsOf: "\r\n".utf8)
|
|
}
|
|
}
|
|
}
|
|
|
|
return data
|
|
}
|
|
|
|
private static func decodeQPLine(_ line: String) -> Data {
|
|
var data = Data()
|
|
var i = line.startIndex
|
|
while i < line.endIndex {
|
|
if line[i] == "=" {
|
|
let next1 = line.index(after: i)
|
|
guard next1 < line.endIndex else {
|
|
data.append(contentsOf: "=".utf8)
|
|
break
|
|
}
|
|
let next2 = line.index(after: next1)
|
|
guard next2 < line.endIndex else {
|
|
data.append(contentsOf: String(line[i...]).utf8)
|
|
break
|
|
}
|
|
let hex = String(line[next1...next2])
|
|
if let byte = UInt8(hex, radix: 16) {
|
|
data.append(byte)
|
|
i = line.index(after: next2)
|
|
} else {
|
|
data.append(contentsOf: "=".utf8)
|
|
i = next1
|
|
}
|
|
} else {
|
|
data.append(contentsOf: String(line[i]).utf8)
|
|
i = line.index(after: i)
|
|
}
|
|
}
|
|
return data
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MIMEParserTests 2>&1 | tail -10
|
|
```
|
|
|
|
Expected: All 11 tests pass.
|
|
|
|
- [ ] **Step 5: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: 111 + 6 + 11 = 128 tests pass (existing + RFC2047 + MIMEParser).
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add Sources/MIMEParser/MIMEParser.swift Tests/MIMEParserTests/MIMEParserTests.swift
|
|
git commit -m "add MIMEParser: multipart parsing, content decoding, boundary generation"
|
|
```
|
|
|
|
---
|
|
|
|
## Chunk 2: Schema Migration, Store Updates & IMAPClient Extensions
|
|
|
|
### Task 5: v4_attachment migration
|
|
|
|
**Files:**
|
|
- Modify: `Sources/MailStore/DatabaseSetup.swift`
|
|
- Modify: `Sources/MailStore/Records/AttachmentRecord.swift`
|
|
- Modify: `Sources/MailStore/Records/MessageRecord.swift`
|
|
- Modify: `Tests/MailStoreTests/MigrationTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing migration test**
|
|
|
|
Open `Tests/MailStoreTests/MigrationTests.swift` and add a test to the existing suite:
|
|
|
|
```swift
|
|
@Test("v4_attachment migration adds sectionPath to attachment and hasAttachments to message")
|
|
func v4AttachmentMigration() throws {
|
|
let db = try DatabaseSetup.openInMemoryDatabase()
|
|
try db.read { db in
|
|
let attachmentColumns = try db.columns(in: "attachment")
|
|
let attachmentColumnNames = attachmentColumns.map(\.name)
|
|
#expect(attachmentColumnNames.contains("sectionPath"))
|
|
|
|
let messageColumns = try db.columns(in: "message")
|
|
let messageColumnNames = messageColumns.map(\.name)
|
|
#expect(messageColumnNames.contains("hasAttachments"))
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MigrationTests/v4AttachmentMigration 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: FAIL — columns don't exist yet.
|
|
|
|
- [ ] **Step 3: Add migration to DatabaseSetup.swift**
|
|
|
|
Add before the `return migrator` line:
|
|
|
|
```swift
|
|
migrator.registerMigration("v4_attachment") { db in
|
|
try db.alter(table: "attachment") { t in
|
|
t.add(column: "sectionPath", .text)
|
|
}
|
|
try db.alter(table: "message") { t in
|
|
t.add(column: "hasAttachments", .boolean).notNull().defaults(to: false)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Update AttachmentRecord to include sectionPath**
|
|
|
|
Add `sectionPath` property to `AttachmentRecord`:
|
|
|
|
```swift
|
|
public struct AttachmentRecord: Codable, FetchableRecord, PersistableRecord, Sendable {
|
|
public static let databaseTableName = "attachment"
|
|
|
|
public var id: String
|
|
public var messageId: String
|
|
public var filename: String?
|
|
public var mimeType: String
|
|
public var size: Int
|
|
public var contentId: String?
|
|
public var cachePath: String?
|
|
public var sectionPath: String?
|
|
|
|
public init(
|
|
id: String, messageId: String, filename: String?, mimeType: String,
|
|
size: Int, contentId: String?, cachePath: String?, sectionPath: String? = nil
|
|
) {
|
|
self.id = id
|
|
self.messageId = messageId
|
|
self.filename = filename
|
|
self.mimeType = mimeType
|
|
self.size = size
|
|
self.contentId = contentId
|
|
self.cachePath = cachePath
|
|
self.sectionPath = sectionPath
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Update MessageRecord to include hasAttachments**
|
|
|
|
Add `hasAttachments` property to `MessageRecord`. Add it after `size`:
|
|
|
|
```swift
|
|
public var hasAttachments: Bool
|
|
```
|
|
|
|
Update the init to include the new parameter with a default value:
|
|
|
|
```swift
|
|
public init(
|
|
id: String, accountId: String, mailboxId: String, uid: Int,
|
|
messageId: String?, inReplyTo: String?, refs: String?,
|
|
subject: String?, fromAddress: String?, fromName: String?,
|
|
toAddresses: String?, ccAddresses: String?,
|
|
date: String, snippet: String?, bodyText: String?, bodyHtml: String?,
|
|
isRead: Bool, isFlagged: Bool, size: Int, hasAttachments: Bool = false
|
|
) {
|
|
// ... existing assignments ...
|
|
self.hasAttachments = hasAttachments
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Run test to verify it passes**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MigrationTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: All migration tests pass.
|
|
|
|
- [ ] **Step 7: Run full test suite to check nothing broke**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass. The new `hasAttachments` field has a default value of `false`, so existing code continues to work.
|
|
|
|
- [ ] **Step 8: Commit**
|
|
|
|
```bash
|
|
git add Sources/MailStore/DatabaseSetup.swift Sources/MailStore/Records/AttachmentRecord.swift Sources/MailStore/Records/MessageRecord.swift Tests/MailStoreTests/MigrationTests.swift
|
|
git commit -m "add v4_attachment migration: sectionPath on attachment, hasAttachments on message"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: MailStore attachment methods & update Queries.swift
|
|
|
|
**Files:**
|
|
- Modify: `Sources/MailStore/MailStore.swift`
|
|
- Modify: `Sources/MailStore/Queries.swift`
|
|
|
|
- [ ] **Step 1: Add attachment CRUD methods to MailStore**
|
|
|
|
Add to MailStore.swift in a new `// MARK: - Attachments` section:
|
|
|
|
```swift
|
|
// MARK: - Attachments
|
|
|
|
public func insertAttachment(_ attachment: AttachmentRecord) throws {
|
|
try dbWriter.write { db in
|
|
try attachment.insert(db)
|
|
}
|
|
}
|
|
|
|
public func insertAttachments(_ attachments: [AttachmentRecord]) throws {
|
|
try dbWriter.write { db in
|
|
for attachment in attachments {
|
|
try attachment.insert(db)
|
|
}
|
|
}
|
|
}
|
|
|
|
public func attachments(messageId: String) throws -> [AttachmentRecord] {
|
|
try dbWriter.read { db in
|
|
try AttachmentRecord
|
|
.filter(Column("messageId") == messageId)
|
|
.order(Column("filename"))
|
|
.fetchAll(db)
|
|
}
|
|
}
|
|
|
|
public func updateAttachmentCachePath(id: String, cachePath: String) throws {
|
|
try dbWriter.write { db in
|
|
try db.execute(
|
|
sql: "UPDATE attachment SET cachePath = ? WHERE id = ?",
|
|
arguments: [cachePath, id]
|
|
)
|
|
}
|
|
}
|
|
|
|
public func updateHasAttachments(messageId: String, hasAttachments: Bool) throws {
|
|
try dbWriter.write { db in
|
|
try db.execute(
|
|
sql: "UPDATE message SET hasAttachments = ? WHERE id = ?",
|
|
arguments: [hasAttachments, messageId]
|
|
)
|
|
}
|
|
}
|
|
|
|
public func storeBodyWithAttachments(
|
|
messageId: String,
|
|
text: String?,
|
|
html: String?,
|
|
attachments: [AttachmentRecord]
|
|
) throws {
|
|
try dbWriter.write { db in
|
|
try db.execute(
|
|
sql: "UPDATE message SET bodyText = ?, bodyHtml = ?, hasAttachments = ? WHERE id = ?",
|
|
arguments: [text, html, !attachments.isEmpty, messageId]
|
|
)
|
|
for attachment in attachments {
|
|
try attachment.insert(db)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Update Queries.swift to read hasAttachments**
|
|
|
|
In `Queries.swift`, update `toMessageSummary` to read from the record:
|
|
|
|
Change:
|
|
```swift
|
|
hasAttachments: false
|
|
```
|
|
To:
|
|
```swift
|
|
hasAttachments: record.hasAttachments
|
|
```
|
|
|
|
- [ ] **Step 3: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Sources/MailStore/MailStore.swift Sources/MailStore/Queries.swift
|
|
git commit -m "add attachment CRUD to MailStore, wire hasAttachments in Queries"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: IMAPClient protocol & implementation — fetchFullMessage, fetchSection
|
|
|
|
**Files:**
|
|
- Modify: `Sources/IMAPClient/IMAPClientProtocol.swift`
|
|
- Modify: `Sources/IMAPClient/IMAPClient.swift`
|
|
- Modify: `Tests/SyncEngineTests/MockIMAPClient.swift`
|
|
|
|
- [ ] **Step 1: Add new methods to IMAPClientProtocol**
|
|
|
|
Add to `IMAPClientProtocol.swift`:
|
|
|
|
```swift
|
|
// v0.5 attachment/MIME operations
|
|
func fetchFullMessage(uid: Int) async throws -> String
|
|
func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data
|
|
```
|
|
|
|
- [ ] **Step 2: Implement fetchFullMessage in IMAPClient**
|
|
|
|
Add to `IMAPClient.swift` in the `// MARK: - Fetch operations` section:
|
|
|
|
```swift
|
|
public func fetchFullMessage(uid: Int) async throws -> String {
|
|
guard var runner else { throw IMAPError.notConnected }
|
|
let uidValue = UID(rawValue: UInt32(uid))
|
|
let range = MessageIdentifierRange<UID>(uidValue...uidValue)
|
|
let set = MessageIdentifierSetNonEmpty<UID>(range: range)
|
|
let responses = try await runner.run(.uidFetch(
|
|
.set(set),
|
|
[.bodySection(peek: true, SectionSpecifier(kind: .complete), nil)],
|
|
[]
|
|
))
|
|
self.runner = runner
|
|
return parseFullMessageResponse(responses)
|
|
}
|
|
```
|
|
|
|
Add the parsing helper:
|
|
|
|
```swift
|
|
private func parseFullMessageResponse(_ responses: [Response]) -> String {
|
|
var bodyBuffer = ByteBuffer()
|
|
for response in responses {
|
|
if case .fetch(let fetchResponse) = response {
|
|
switch fetchResponse {
|
|
case .streamingBytes(let bytes):
|
|
var mutableBytes = bytes
|
|
bodyBuffer.writeBuffer(&mutableBytes)
|
|
default:
|
|
break
|
|
}
|
|
}
|
|
}
|
|
return String(buffer: bodyBuffer)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Implement fetchSection in IMAPClient**
|
|
|
|
Add to `IMAPClient.swift`:
|
|
|
|
```swift
|
|
public func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data {
|
|
guard var runner else { throw IMAPError.notConnected }
|
|
|
|
// Select the mailbox first
|
|
let selectResponses = try await runner.run(.select(MailboxName(ByteBuffer(string: mailbox))))
|
|
self.runner = runner
|
|
guard selectResponses.contains(where: { isOKTagged($0) }) else {
|
|
throw IMAPError.unexpectedResponse("SELECT \(mailbox) failed")
|
|
}
|
|
|
|
let uidValue = UID(rawValue: UInt32(uid))
|
|
let range = MessageIdentifierRange<UID>(uidValue...uidValue)
|
|
let set = MessageIdentifierSetNonEmpty<UID>(range: range)
|
|
|
|
// Parse section path into SectionSpecifier.Part
|
|
let sectionParts = section.split(separator: ".").compactMap { Int($0) }
|
|
let part = SectionSpecifier.Part(sectionParts)
|
|
let spec = SectionSpecifier(part: part)
|
|
|
|
let responses = try await runner.run(.uidFetch(
|
|
.set(set),
|
|
[.bodySection(peek: true, spec, nil)],
|
|
[]
|
|
))
|
|
self.runner = runner
|
|
|
|
var bodyBuffer = ByteBuffer()
|
|
for response in responses {
|
|
if case .fetch(let fetchResponse) = response {
|
|
if case .streamingBytes(let bytes) = fetchResponse {
|
|
var mutableBytes = bytes
|
|
bodyBuffer.writeBuffer(&mutableBytes)
|
|
}
|
|
}
|
|
}
|
|
|
|
// The section content may be base64 encoded — decode it
|
|
let raw = String(buffer: bodyBuffer)
|
|
let cleaned = raw.filter { !$0.isWhitespace }
|
|
if let decoded = Data(base64Encoded: cleaned) {
|
|
return decoded
|
|
}
|
|
return Data(bodyBuffer.readableBytesView)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Update MockIMAPClient**
|
|
|
|
Add to `MockIMAPClient.swift`:
|
|
|
|
Properties:
|
|
```swift
|
|
var fullMessages: [Int: String] = [:]
|
|
var sections: [String: Data] = [:] // key: "uid-section"
|
|
var fetchFullMessageCalls: [Int] = []
|
|
var fetchSectionCalls: [(uid: Int, mailbox: String, section: String)] = []
|
|
```
|
|
|
|
Methods:
|
|
```swift
|
|
func fetchFullMessage(uid: Int) async throws -> String {
|
|
fetchFullMessageCalls.append(uid)
|
|
return fullMessages[uid] ?? ""
|
|
}
|
|
|
|
func fetchSection(uid: Int, mailbox: String, section: String) async throws -> Data {
|
|
fetchSectionCalls.append((uid: uid, mailbox: mailbox, section: section))
|
|
return sections["\(uid)-\(section)"] ?? Data()
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Build to verify compilation**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build
|
|
```
|
|
|
|
Expected: Build succeeds.
|
|
|
|
- [ ] **Step 6: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add Sources/IMAPClient/IMAPClientProtocol.swift Sources/IMAPClient/IMAPClient.swift Tests/SyncEngineTests/MockIMAPClient.swift
|
|
git commit -m "add fetchFullMessage, fetchSection to IMAPClient for MIME attachment retrieval"
|
|
```
|
|
|
|
---
|
|
|
|
## Chunk 3: MessageFormatter Multipart & SyncCoordinator MIME Integration
|
|
|
|
### Task 8: MessageFormatter — multipart formatting & base64 encoding
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SMTPClient/MessageFormatter.swift`
|
|
- Modify: `Tests/SMTPClientTests/MessageFormatterTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing tests for multipart formatting**
|
|
|
|
Add to `MessageFormatterTests.swift`:
|
|
|
|
```swift
|
|
// MARK: - Multipart Formatting
|
|
|
|
@Test("format with attachments produces multipart/mixed")
|
|
func multipartWithAttachment() {
|
|
let message = OutgoingMessage(
|
|
from: EmailAddress(address: "alice@example.com"),
|
|
to: [EmailAddress(address: "bob@example.com")],
|
|
subject: "With attachment",
|
|
bodyText: "See attached.",
|
|
messageId: "test@example.com"
|
|
)
|
|
let attachments: [(filename: String, mimeType: String, data: Data)] = [
|
|
(filename: "test.pdf", mimeType: "application/pdf", data: Data("PDF content".utf8)),
|
|
]
|
|
|
|
let formatted = MessageFormatter.formatMultipart(message, attachments: attachments)
|
|
|
|
#expect(formatted.contains("Content-Type: multipart/mixed; boundary="))
|
|
#expect(formatted.contains("Content-Type: text/plain; charset=utf-8"))
|
|
#expect(formatted.contains("Content-Type: application/pdf; name=\"test.pdf\""))
|
|
#expect(formatted.contains("Content-Disposition: attachment; filename=\"test.pdf\""))
|
|
#expect(formatted.contains("Content-Transfer-Encoding: base64"))
|
|
#expect(formatted.contains("See attached."))
|
|
}
|
|
|
|
@Test("format without attachments produces single-part")
|
|
func singlePartNoAttachments() {
|
|
let message = OutgoingMessage(
|
|
from: EmailAddress(address: "alice@example.com"),
|
|
to: [EmailAddress(address: "bob@example.com")],
|
|
subject: "Plain",
|
|
bodyText: "Just text.",
|
|
messageId: "test@example.com"
|
|
)
|
|
|
|
let formatted = MessageFormatter.format(message)
|
|
|
|
#expect(formatted.contains("Content-Type: text/plain; charset=utf-8"))
|
|
#expect(!formatted.contains("multipart"))
|
|
}
|
|
|
|
@Test("base64 encoding wraps at 76 characters")
|
|
func base64LineWrapping() {
|
|
let data = Data(repeating: 0xFF, count: 100)
|
|
let encoded = MessageFormatter.base64Encode(data)
|
|
let lines = encoded.components(separatedBy: "\r\n")
|
|
for line in lines where !line.isEmpty {
|
|
#expect(line.count <= 76)
|
|
}
|
|
}
|
|
|
|
@Test("multiple attachments produce correct number of boundary sections")
|
|
func multipleAttachments() {
|
|
let message = OutgoingMessage(
|
|
from: EmailAddress(address: "alice@example.com"),
|
|
to: [EmailAddress(address: "bob@example.com")],
|
|
subject: "Multi",
|
|
bodyText: "Files.",
|
|
messageId: "test@example.com"
|
|
)
|
|
let attachments: [(filename: String, mimeType: String, data: Data)] = [
|
|
(filename: "a.pdf", mimeType: "application/pdf", data: Data("A".utf8)),
|
|
(filename: "b.jpg", mimeType: "image/jpeg", data: Data("B".utf8)),
|
|
]
|
|
|
|
let formatted = MessageFormatter.formatMultipart(message, attachments: attachments)
|
|
|
|
#expect(formatted.contains("a.pdf"))
|
|
#expect(formatted.contains("b.jpg"))
|
|
#expect(formatted.contains("application/pdf"))
|
|
#expect(formatted.contains("image/jpeg"))
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MessageFormatterTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: Compilation error — `formatMultipart` and `base64Encode` not defined.
|
|
|
|
- [ ] **Step 3: Implement formatMultipart and base64Encode**
|
|
|
|
Add to `MessageFormatter.swift`:
|
|
|
|
```swift
|
|
/// Format a multipart/mixed message with attachments.
|
|
public static func formatMultipart(
|
|
_ message: OutgoingMessage,
|
|
attachments: [(filename: String, mimeType: String, data: Data)]
|
|
) -> String {
|
|
let boundary = "=_MagnumOpus_\(UUID().uuidString)"
|
|
|
|
var headers: [(String, String)] = []
|
|
headers.append(("From", formatAddress(message.from)))
|
|
headers.append(("To", message.to.map(formatAddress).joined(separator: ", ")))
|
|
|
|
if !message.cc.isEmpty {
|
|
headers.append(("Cc", message.cc.map(formatAddress).joined(separator: ", ")))
|
|
}
|
|
|
|
headers.append(("Subject", message.subject))
|
|
headers.append(("Date", formatRFC2822Date(Date())))
|
|
headers.append(("Message-ID", "<\(message.messageId)>"))
|
|
|
|
if let inReplyTo = message.inReplyTo {
|
|
headers.append(("In-Reply-To", "<\(inReplyTo)>"))
|
|
}
|
|
if let references = message.references {
|
|
headers.append(("References", references))
|
|
}
|
|
|
|
headers.append(("MIME-Version", "1.0"))
|
|
headers.append(("Content-Type", "multipart/mixed; boundary=\"\(boundary)\""))
|
|
|
|
var result = ""
|
|
for (name, value) in headers {
|
|
result += "\(name): \(value)\r\n"
|
|
}
|
|
result += "\r\n"
|
|
|
|
// Text body part
|
|
result += "--\(boundary)\r\n"
|
|
result += "Content-Type: text/plain; charset=utf-8\r\n"
|
|
result += "Content-Transfer-Encoding: quoted-printable\r\n"
|
|
result += "\r\n"
|
|
result += quotedPrintableEncode(message.bodyText)
|
|
result += "\r\n"
|
|
|
|
// Attachment parts
|
|
for attachment in attachments {
|
|
result += "--\(boundary)\r\n"
|
|
result += "Content-Type: \(attachment.mimeType); name=\"\(attachment.filename)\"\r\n"
|
|
result += "Content-Disposition: attachment; filename=\"\(attachment.filename)\"\r\n"
|
|
result += "Content-Transfer-Encoding: base64\r\n"
|
|
result += "\r\n"
|
|
result += base64Encode(attachment.data)
|
|
result += "\r\n"
|
|
}
|
|
|
|
// Closing boundary
|
|
result += "--\(boundary)--\r\n"
|
|
|
|
return result
|
|
}
|
|
|
|
/// Base64 encode data with line wrapping at 76 characters per RFC 2045.
|
|
public static func base64Encode(_ data: Data, lineLength: Int = 76) -> String {
|
|
let encoded = data.base64EncodedString()
|
|
var result = ""
|
|
var index = encoded.startIndex
|
|
while index < encoded.endIndex {
|
|
let end = encoded.index(index, offsetBy: lineLength, limitedBy: encoded.endIndex) ?? encoded.endIndex
|
|
result += String(encoded[index..<end])
|
|
if end < encoded.endIndex {
|
|
result += "\r\n"
|
|
}
|
|
index = end
|
|
}
|
|
return result
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MessageFormatterTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Sources/SMTPClient/MessageFormatter.swift Tests/SMTPClientTests/MessageFormatterTests.swift
|
|
git commit -m "add multipart/mixed formatting, base64 line-wrapped encoding to MessageFormatter"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 9: SyncCoordinator MIME-aware body prefetch
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SyncEngine/SyncCoordinator.swift`
|
|
- Modify: `Package.swift` (add MIMEParser dependency to SyncEngine)
|
|
- Modify: `Tests/SyncEngineTests/SyncCoordinatorTests.swift`
|
|
|
|
- [ ] **Step 1: Add MIMEParser dependency to SyncEngine target**
|
|
|
|
In `Package.swift`, update the SyncEngine target dependencies:
|
|
|
|
```swift
|
|
.target(
|
|
name: "SyncEngine",
|
|
dependencies: ["Models", "IMAPClient", "SMTPClient", "MailStore", "TaskStore", "MIMEParser"]
|
|
),
|
|
```
|
|
|
|
- [ ] **Step 2: Write failing test for MIME-aware sync**
|
|
|
|
Add to `SyncCoordinatorTests.swift`:
|
|
|
|
```swift
|
|
@Test("sync with full message parses MIME and stores attachments")
|
|
func mimeAwareSync() async throws {
|
|
let store = try makeStore()
|
|
let mock = makeMock()
|
|
|
|
// Provide a multipart MIME message for body prefetch
|
|
let mimeMessage = """
|
|
Content-Type: multipart/mixed; boundary="bound"\r
|
|
\r
|
|
--bound\r
|
|
Content-Type: text/plain; charset=utf-8\r
|
|
\r
|
|
Hello from MIME.\r
|
|
--bound\r
|
|
Content-Type: application/pdf; name="report.pdf"\r
|
|
Content-Disposition: attachment; filename="report.pdf"\r
|
|
Content-Transfer-Encoding: base64\r
|
|
\r
|
|
SGVsbG8=\r
|
|
--bound--
|
|
"""
|
|
mock.fullMessages[1] = mimeMessage
|
|
|
|
let coordinator = SyncCoordinator(
|
|
accountConfig: AccountConfig(
|
|
id: "acc1", name: "Personal", email: "me@example.com",
|
|
imapHost: "imap.example.com", imapPort: 993
|
|
),
|
|
imapClient: mock,
|
|
store: store
|
|
)
|
|
|
|
try await coordinator.syncNow()
|
|
|
|
// Verify MIME body was stored
|
|
let inboxMb = try store.mailboxes(accountId: "acc1").first { $0.name == "INBOX" }!
|
|
let messages = try store.messages(mailboxId: inboxMb.id)
|
|
let msg = messages.first { $0.uid == 1 }!
|
|
#expect(msg.bodyText == "Hello from MIME.")
|
|
#expect(msg.hasAttachments == true)
|
|
|
|
// Verify attachment was stored
|
|
let attachments = try store.attachments(messageId: msg.id)
|
|
#expect(attachments.count == 1)
|
|
#expect(attachments.first?.filename == "report.pdf")
|
|
#expect(attachments.first?.mimeType == "application/pdf")
|
|
#expect(attachments.first?.sectionPath == "2")
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Run test to verify it fails**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter SyncCoordinatorTests/mimeAwareSync 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: FAIL — `fullMessages` not in mock or body not parsed as MIME.
|
|
|
|
- [ ] **Step 4: Update SyncCoordinator.prefetchBodies to use MIME parsing**
|
|
|
|
Replace the `prefetchBodies` method in `SyncCoordinator.swift`:
|
|
|
|
```swift
|
|
import MIMEParser
|
|
```
|
|
|
|
(Add at the top of the file with other imports.)
|
|
|
|
```swift
|
|
/// Fetch full RFC822 messages and parse MIME for body + attachments
|
|
private func prefetchBodies(mailboxId: String) async {
|
|
let thirtyDaysAgo = ISO8601DateFormatter().string(
|
|
from: Calendar.current.date(byAdding: .day, value: -30, to: Date())!
|
|
)
|
|
do {
|
|
let messages = try store.messages(mailboxId: mailboxId)
|
|
let recent = messages.filter { $0.bodyText == nil && $0.bodyHtml == nil && $0.date >= thirtyDaysAgo }
|
|
for message in recent.prefix(50) {
|
|
guard !Task.isCancelled else { break }
|
|
let rawMessage = try await imapClient.fetchFullMessage(uid: message.uid)
|
|
guard !rawMessage.isEmpty else { continue }
|
|
|
|
let parsed = MIMEParser.parse(rawMessage)
|
|
|
|
// Build attachment records
|
|
let attachmentRecords = (parsed.attachments + parsed.inlineImages).map { att in
|
|
AttachmentRecord(
|
|
id: UUID().uuidString,
|
|
messageId: message.id,
|
|
filename: att.filename,
|
|
mimeType: att.mimeType,
|
|
size: att.size,
|
|
contentId: att.contentId,
|
|
cachePath: nil,
|
|
sectionPath: att.sectionPath
|
|
)
|
|
}
|
|
|
|
try store.storeBodyWithAttachments(
|
|
messageId: message.id,
|
|
text: parsed.textBody,
|
|
html: parsed.htmlBody,
|
|
attachments: attachmentRecords
|
|
)
|
|
}
|
|
} catch {
|
|
// Background prefetch failure is non-fatal
|
|
print("[SyncCoordinator] prefetchBodies error: \(error)")
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Run test to verify it passes**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter SyncCoordinatorTests/mimeAwareSync 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 6: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add Package.swift Sources/SyncEngine/SyncCoordinator.swift Tests/SyncEngineTests/SyncCoordinatorTests.swift
|
|
git commit -m "replace body text prefetch with MIME-aware parsing, store attachments from parsed messages"
|
|
```
|
|
|
|
---
|
|
|
|
## Chunk 4: IMAP IDLE
|
|
|
|
### Task 10: IMAPIdleHandler — NIO channel handler for IDLE responses
|
|
|
|
**Files:**
|
|
- Create: `Sources/IMAPClient/IMAPIdleHandler.swift`
|
|
|
|
- [ ] **Step 1: Write IMAPIdleHandler**
|
|
|
|
```swift
|
|
import NIO
|
|
import NIOIMAPCore
|
|
import NIOIMAP
|
|
|
|
/// Events emitted by the IDLE handler
|
|
public enum IMAPIdleEvent: Sendable {
|
|
case exists(Int)
|
|
case expunge(Int)
|
|
case idleTerminated
|
|
}
|
|
|
|
/// NIO ChannelInboundHandler that processes untagged responses during IMAP IDLE.
|
|
/// Unlike the standard IMAPResponseHandler (which uses CheckedContinuation for tagged responses),
|
|
/// this handler uses AsyncStream to deliver a continuous stream of events.
|
|
final class IMAPIdleHandler: ChannelInboundHandler, @unchecked Sendable {
|
|
typealias InboundIn = Response
|
|
|
|
private let continuation: AsyncStream<IMAPIdleEvent>.Continuation
|
|
private var idleTag: String?
|
|
|
|
init(continuation: AsyncStream<IMAPIdleEvent>.Continuation) {
|
|
self.continuation = continuation
|
|
}
|
|
|
|
func setIdleTag(_ tag: String) {
|
|
idleTag = tag
|
|
}
|
|
|
|
func channelRead(context: ChannelHandlerContext, data: NIOAny) {
|
|
let response = unwrapInboundIn(data)
|
|
|
|
switch response {
|
|
case .untagged(let payload):
|
|
switch payload {
|
|
case .mailboxData(let data):
|
|
switch data {
|
|
case .exists(let count):
|
|
continuation.yield(.exists(count))
|
|
default:
|
|
break
|
|
}
|
|
case .messageData(let data):
|
|
switch data {
|
|
case .expunge(let seqNum):
|
|
continuation.yield(.expunge(seqNum))
|
|
default:
|
|
break
|
|
}
|
|
default:
|
|
break
|
|
}
|
|
case .tagged(let tagged):
|
|
if tagged.tag == idleTag {
|
|
continuation.yield(.idleTerminated)
|
|
}
|
|
case .idleStarted:
|
|
// Server acknowledged IDLE — we're now idling
|
|
break
|
|
case .fetch, .authenticationChallenge, .fatal:
|
|
break
|
|
}
|
|
}
|
|
|
|
func errorCaught(context: ChannelHandlerContext, error: Error) {
|
|
continuation.finish()
|
|
context.close(promise: nil)
|
|
}
|
|
|
|
func channelInactive(context: ChannelHandlerContext) {
|
|
continuation.finish()
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Verify build**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build
|
|
```
|
|
|
|
Expected: Build succeeds.
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add Sources/IMAPClient/IMAPIdleHandler.swift
|
|
git commit -m "add IMAPIdleHandler: NIO channel handler for IDLE event streaming"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: IMAPIdleClient actor
|
|
|
|
**Files:**
|
|
- Create: `Sources/IMAPClient/IMAPIdleClient.swift`
|
|
- Create: `Tests/IMAPClientTests/IMAPIdleClientTests.swift`
|
|
|
|
- [ ] **Step 1: Write IMAPIdleClient**
|
|
|
|
```swift
|
|
import Foundation
|
|
import NIO
|
|
import NIOIMAPCore
|
|
@preconcurrency import NIOIMAP
|
|
@preconcurrency import NIOSSL
|
|
import Models
|
|
|
|
public actor IMAPIdleClient {
|
|
private let host: String
|
|
private let port: Int
|
|
private let credentials: Credentials
|
|
private var channel: Channel?
|
|
private var group: EventLoopGroup?
|
|
private var isMonitoring = false
|
|
private var monitorTask: Task<Void, Never>?
|
|
private let reIdleInterval: Duration = .seconds(29 * 60) // 29 minutes per RFC 2177
|
|
|
|
public init(host: String, port: Int, credentials: Credentials) {
|
|
self.host = host
|
|
self.port = port
|
|
self.credentials = credentials
|
|
}
|
|
|
|
/// Start monitoring INBOX via IMAP IDLE. Calls onNewMail when server sends EXISTS.
|
|
public func startMonitoring(onNewMail: @escaping @Sendable () -> Void) async throws {
|
|
guard !isMonitoring else { return }
|
|
isMonitoring = true
|
|
|
|
try await connectAndLogin()
|
|
try await selectInbox()
|
|
|
|
monitorTask = Task { [weak self] in
|
|
var backoffSeconds: UInt64 = 5
|
|
while !Task.isCancelled {
|
|
guard let self else { break }
|
|
do {
|
|
try await self.idleLoop(onNewMail: onNewMail)
|
|
} catch {
|
|
if Task.isCancelled { break }
|
|
// Reconnect with exponential backoff
|
|
try? await Task.sleep(for: .seconds(Int(backoffSeconds)))
|
|
backoffSeconds = min(backoffSeconds * 2, 300) // cap at 5 min
|
|
do {
|
|
try await self.connectAndLogin()
|
|
try await self.selectInbox()
|
|
backoffSeconds = 5 // reset on success
|
|
} catch {
|
|
if Task.isCancelled { break }
|
|
continue
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Stop monitoring and disconnect.
|
|
public func stopMonitoring() async {
|
|
isMonitoring = false
|
|
monitorTask?.cancel()
|
|
monitorTask = nil
|
|
try? await channel?.close()
|
|
channel = nil
|
|
try? await group?.shutdownGracefully()
|
|
group = nil
|
|
}
|
|
|
|
// MARK: - Connection
|
|
|
|
private func connectAndLogin() async throws {
|
|
let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
|
|
let sslContext = try NIOSSLContext(configuration: TLSConfiguration.makeClientConfiguration())
|
|
let hostname = host
|
|
|
|
let responseHandler = IMAPResponseHandler()
|
|
|
|
let bootstrap = ClientBootstrap(group: eventLoopGroup)
|
|
.channelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
|
|
.channelInitializer { channel in
|
|
let sslHandler = try! NIOSSLClientHandler(context: sslContext, serverHostname: hostname)
|
|
return channel.pipeline.addHandlers([
|
|
sslHandler,
|
|
IMAPClientHandler(),
|
|
responseHandler,
|
|
])
|
|
}
|
|
|
|
let chan = try await bootstrap.connect(host: host, port: port).get()
|
|
try await responseHandler.waitForGreeting()
|
|
|
|
// Login
|
|
var tagCounter = 0
|
|
func nextTag() -> String {
|
|
tagCounter += 1
|
|
return "IDLE\(tagCounter)"
|
|
}
|
|
|
|
let loginTag = nextTag()
|
|
let loginCommand = TaggedCommand(
|
|
tag: loginTag,
|
|
command: .login(username: credentials.username, password: credentials.password)
|
|
)
|
|
let loginResponses = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[Response], Error>) in
|
|
responseHandler.sendCommand(tag: loginTag, continuation: cont)
|
|
chan.writeAndFlush(IMAPClientHandler.Message.part(.tagged(loginCommand)), promise: nil)
|
|
}
|
|
guard loginResponses.contains(where: { isOK($0) }) else {
|
|
throw IMAPError.authenticationFailed
|
|
}
|
|
|
|
self.channel = chan
|
|
self.group = eventLoopGroup
|
|
}
|
|
|
|
private func selectInbox() async throws {
|
|
guard let channel else { throw IMAPError.notConnected }
|
|
|
|
let responseHandler = try await getResponseHandler()
|
|
let tag = "IDLESEL1"
|
|
let selectCommand = TaggedCommand(
|
|
tag: tag,
|
|
command: .select(MailboxName(ByteBuffer(string: "INBOX")))
|
|
)
|
|
let responses = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[Response], Error>) in
|
|
responseHandler.sendCommand(tag: tag, continuation: cont)
|
|
channel.writeAndFlush(IMAPClientHandler.Message.part(.tagged(selectCommand)), promise: nil)
|
|
}
|
|
guard responses.contains(where: { isOK($0) }) else {
|
|
throw IMAPError.unexpectedResponse("SELECT INBOX failed")
|
|
}
|
|
}
|
|
|
|
// MARK: - IDLE Loop
|
|
|
|
private func idleLoop(onNewMail: @escaping @Sendable () -> Void) async throws {
|
|
guard let channel else { throw IMAPError.notConnected }
|
|
let pipeline = channel.pipeline
|
|
|
|
// Iterative IDLE loop — avoids unbounded stack growth from recursion
|
|
while !Task.isCancelled {
|
|
// Swap response handler for IDLE handler
|
|
let (stream, streamContinuation) = AsyncStream<IMAPIdleEvent>.makeStream()
|
|
let idleHandler = IMAPIdleHandler(continuation: streamContinuation)
|
|
|
|
let oldHandler = try await getResponseHandler()
|
|
try await pipeline.removeHandler(oldHandler).get()
|
|
try await pipeline.addHandler(idleHandler).get()
|
|
|
|
let idleTag = "IDLE1"
|
|
idleHandler.setIdleTag(idleTag)
|
|
|
|
// Send IDLE command
|
|
let idleCommand = TaggedCommand(tag: idleTag, command: .idleStart)
|
|
channel.writeAndFlush(IMAPClientHandler.Message.part(.tagged(idleCommand)), promise: nil)
|
|
|
|
// Set up re-IDLE timer — after 29 min, send DONE to break IDLE (RFC 2177)
|
|
let reIdleChannel = channel
|
|
let reIdleContinuation = streamContinuation
|
|
let reIdleTask = Task {
|
|
try await Task.sleep(for: reIdleInterval)
|
|
// Timer fired — break IDLE so the outer loop re-enters
|
|
reIdleChannel.writeAndFlush(IMAPClientHandler.Message.part(.idleDone), promise: nil)
|
|
}
|
|
|
|
var shouldReIdle = false
|
|
|
|
// Consume events
|
|
for await event in stream {
|
|
switch event {
|
|
case .exists:
|
|
// New mail — break IDLE, trigger sync
|
|
channel.writeAndFlush(IMAPClientHandler.Message.part(.idleDone), promise: nil)
|
|
for await innerEvent in stream {
|
|
if case .idleTerminated = innerEvent { break }
|
|
}
|
|
reIdleTask.cancel()
|
|
onNewMail()
|
|
shouldReIdle = true
|
|
|
|
case .expunge:
|
|
break
|
|
|
|
case .idleTerminated:
|
|
reIdleTask.cancel()
|
|
shouldReIdle = true
|
|
}
|
|
if shouldReIdle { break }
|
|
}
|
|
|
|
// Restore response handler before re-entering IDLE
|
|
try await pipeline.removeHandler(idleHandler).get()
|
|
let newResponseHandler = IMAPResponseHandler()
|
|
try await pipeline.addHandler(newResponseHandler).get()
|
|
|
|
if !shouldReIdle {
|
|
// Stream ended — connection dropped
|
|
throw IMAPError.serverError("IDLE connection dropped")
|
|
}
|
|
// Otherwise: loop back to re-enter IDLE
|
|
}
|
|
}
|
|
|
|
// MARK: - Helpers
|
|
|
|
private func getResponseHandler() async throws -> IMAPResponseHandler {
|
|
guard let channel else { throw IMAPError.notConnected }
|
|
return try await channel.pipeline.handler(type: IMAPResponseHandler.self).get()
|
|
}
|
|
|
|
private func isOK(_ response: Response) -> Bool {
|
|
if case .tagged(let tagged) = response {
|
|
if case .ok = tagged.state { return true }
|
|
}
|
|
return false
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Write basic IDLE client tests**
|
|
|
|
```swift
|
|
import Testing
|
|
import Foundation
|
|
@testable import IMAPClient
|
|
import Models
|
|
|
|
@Suite("IMAPIdleClient")
|
|
struct IMAPIdleClientTests {
|
|
|
|
@Test("IMAPIdleClient can be initialized")
|
|
func initialization() {
|
|
let client = IMAPIdleClient(
|
|
host: "imap.example.com",
|
|
port: 993,
|
|
credentials: Credentials(username: "user", password: "pass")
|
|
)
|
|
// Just verify it compiles and initializes — actual IDLE testing
|
|
// requires a real server or more sophisticated mocking
|
|
#expect(true)
|
|
}
|
|
|
|
@Test("IMAPIdleEvent cases exist")
|
|
func eventCases() {
|
|
let exists = IMAPIdleEvent.exists(42)
|
|
let expunge = IMAPIdleEvent.expunge(1)
|
|
let terminated = IMAPIdleEvent.idleTerminated
|
|
|
|
if case .exists(let count) = exists {
|
|
#expect(count == 42)
|
|
}
|
|
if case .expunge(let num) = expunge {
|
|
#expect(num == 1)
|
|
}
|
|
if case .idleTerminated = terminated {
|
|
#expect(true)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Run tests**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter IMAPIdleClientTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: Tests pass.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Sources/IMAPClient/IMAPIdleClient.swift Tests/IMAPClientTests/IMAPIdleClientTests.swift
|
|
git commit -m "add IMAPIdleClient actor with IDLE loop, reconnect backoff, re-IDLE timer"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 12: SyncCoordinator IDLE integration
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SyncEngine/SyncCoordinator.swift`
|
|
- Modify: `Tests/SyncEngineTests/SyncCoordinatorTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing test for IDLE integration**
|
|
|
|
Add to `SyncCoordinatorTests.swift`:
|
|
|
|
```swift
|
|
@Test("startIdleMonitoring does nothing when server lacks IDLE capability")
|
|
func idleNotSupported() async throws {
|
|
let store = try makeStore()
|
|
let mock = makeMock()
|
|
mock.serverCapabilities = ["IMAP4rev1"] // No IDLE
|
|
|
|
let coordinator = SyncCoordinator(
|
|
accountConfig: AccountConfig(
|
|
id: "acc1", name: "Personal", email: "me@example.com",
|
|
imapHost: "imap.example.com", imapPort: 993
|
|
),
|
|
imapClient: mock,
|
|
store: store
|
|
)
|
|
|
|
// First sync to establish account
|
|
try await coordinator.syncNow()
|
|
|
|
// Should not crash, just return without starting IDLE
|
|
await coordinator.startIdleMonitoring()
|
|
|
|
// Verify no IDLE client was created (by checking that sync still works normally)
|
|
try await coordinator.syncNow()
|
|
#expect(coordinator.syncState == .idle)
|
|
}
|
|
|
|
@Test("stopIdleMonitoring is safe to call even when not monitoring")
|
|
func stopIdleWhenNotMonitoring() async throws {
|
|
let store = try makeStore()
|
|
let mock = makeMock()
|
|
let coordinator = SyncCoordinator(
|
|
accountConfig: AccountConfig(
|
|
id: "acc1", name: "Personal", email: "me@example.com",
|
|
imapHost: "imap.example.com", imapPort: 993
|
|
),
|
|
imapClient: mock,
|
|
store: store
|
|
)
|
|
|
|
// Should not crash
|
|
await coordinator.stopIdleMonitoring()
|
|
#expect(coordinator.syncState == .idle)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter "SyncCoordinatorTests/idleNotSupported|SyncCoordinatorTests/stopIdleWhenNotMonitoring" 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: Compilation error — methods don't exist.
|
|
|
|
- [ ] **Step 3: Add IDLE integration to SyncCoordinator**
|
|
|
|
First, update the `SyncCoordinator` init to accept `Credentials` for IDLE use. Add a new stored property:
|
|
|
|
```swift
|
|
private let credentials: Credentials?
|
|
```
|
|
|
|
Update the init signature (add `credentials` parameter with default `nil`):
|
|
|
|
```swift
|
|
public init(
|
|
accountConfig: AccountConfig,
|
|
imapClient: any IMAPClientProtocol,
|
|
store: MailStore,
|
|
actionQueue: ActionQueue? = nil,
|
|
taskStore: TaskStore? = nil,
|
|
credentials: Credentials? = nil
|
|
) {
|
|
self.accountConfig = accountConfig
|
|
self.imapClient = imapClient
|
|
self.store = store
|
|
self.actionQueue = actionQueue
|
|
self.taskStore = taskStore
|
|
self.credentials = credentials
|
|
}
|
|
```
|
|
|
|
Add IDLE properties after existing ones:
|
|
|
|
```swift
|
|
private var idleClient: IMAPIdleClient?
|
|
private var idleActive = false
|
|
```
|
|
|
|
Add IDLE methods:
|
|
|
|
```swift
|
|
// MARK: - IMAP IDLE
|
|
|
|
/// Check server capabilities and start IDLE monitoring if supported.
|
|
/// Must be called after at least one successful sync (so capabilities are cached).
|
|
public func startIdleMonitoring() async {
|
|
guard let credentials else {
|
|
print("[SyncCoordinator] No credentials provided, cannot start IDLE")
|
|
return
|
|
}
|
|
|
|
do {
|
|
let caps = try await imapClient.capabilities()
|
|
guard caps.contains("IDLE") else {
|
|
print("[SyncCoordinator] Server does not support IDLE, using periodic sync only")
|
|
return
|
|
}
|
|
|
|
let client = IMAPIdleClient(
|
|
host: accountConfig.imapHost,
|
|
port: accountConfig.imapPort,
|
|
credentials: credentials
|
|
)
|
|
|
|
idleClient = client
|
|
idleActive = true
|
|
|
|
try await client.startMonitoring { [weak self] in
|
|
Task { @MainActor [weak self] in
|
|
try? await self?.syncNow()
|
|
}
|
|
}
|
|
} catch {
|
|
print("[SyncCoordinator] Failed to start IDLE monitoring: \(error)")
|
|
}
|
|
}
|
|
|
|
/// Stop IDLE monitoring and clean up.
|
|
public func stopIdleMonitoring() async {
|
|
idleActive = false
|
|
await idleClient?.stopMonitoring()
|
|
idleClient = nil
|
|
}
|
|
```
|
|
|
|
**Note:** Existing callers that don't pass `credentials` continue to work — `credentials` defaults to `nil` and IDLE simply won't start. The app layer passes credentials when constructing SyncCoordinator.
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter SyncCoordinatorTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: All SyncCoordinator tests pass.
|
|
|
|
- [ ] **Step 5: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add Sources/SyncEngine/SyncCoordinator.swift Tests/SyncEngineTests/SyncCoordinatorTests.swift
|
|
git commit -m "add IDLE monitoring to SyncCoordinator: start/stop with capability check"
|
|
```
|
|
|
|
---
|
|
|
|
## Chunk 5: SyncEngine Update — Stop Fetching Body in Envelopes
|
|
|
|
### Task 13: Remove body fetch from fetchEnvelopes
|
|
|
|
The existing `fetchEnvelopes` fetches body text inline during initial sync via `BODY[TEXT]`. Now that `prefetchBodies` uses `fetchFullMessage` with MIME parsing, we should stop fetching bodies in `fetchEnvelopes` to avoid doubling bandwidth. The body data from envelopes was being stored as raw text without MIME parsing anyway, so it's better to let the MIME-aware `prefetchBodies` handle it.
|
|
|
|
**Files:**
|
|
- Modify: `Sources/IMAPClient/IMAPClient.swift`
|
|
- Modify: `Sources/IMAPClient/FetchedEnvelope.swift`
|
|
- Modify: `Sources/SyncEngine/SyncCoordinator.swift`
|
|
|
|
- [ ] **Step 1: Remove BODY[TEXT] from fetchEnvelopes fetch attributes**
|
|
|
|
In `IMAPClient.swift`, update `fetchEnvelopes` — remove the `.bodySection` attribute:
|
|
|
|
Change:
|
|
```swift
|
|
let responses = try await runner.run(.uidFetch(
|
|
.set(set),
|
|
[
|
|
.envelope,
|
|
.flags,
|
|
.uid,
|
|
.rfc822Size,
|
|
.bodySection(peek: true, SectionSpecifier(kind: .text), nil),
|
|
],
|
|
[]
|
|
))
|
|
```
|
|
|
|
To:
|
|
```swift
|
|
let responses = try await runner.run(.uidFetch(
|
|
.set(set),
|
|
[
|
|
.envelope,
|
|
.flags,
|
|
.uid,
|
|
.rfc822Size,
|
|
],
|
|
[]
|
|
))
|
|
```
|
|
|
|
- [ ] **Step 2: Simplify parseFetchResponses — remove bodyBuffer**
|
|
|
|
In `IMAPClient.swift`, update `parseFetchResponses` to remove `bodyBuffer`. The body now comes from MIME-aware `prefetchBodies`, not from envelope fetch.
|
|
|
|
Replace the method:
|
|
|
|
```swift
|
|
private func parseFetchResponses(_ responses: [Response]) -> [FetchedEnvelope] {
|
|
var envelopes: [FetchedEnvelope] = []
|
|
var currentUID: Int?
|
|
var currentEnvelope: Envelope?
|
|
var currentFlags: [Flag] = []
|
|
var currentSize: Int = 0
|
|
|
|
for response in responses {
|
|
switch response {
|
|
case .fetch(let fetchResponse):
|
|
switch fetchResponse {
|
|
case .start, .startUID:
|
|
currentUID = nil
|
|
currentEnvelope = nil
|
|
currentFlags = []
|
|
currentSize = 0
|
|
case .simpleAttribute(let attr):
|
|
switch attr {
|
|
case .uid(let uid):
|
|
currentUID = Int(uid.rawValue)
|
|
case .envelope(let env):
|
|
currentEnvelope = env
|
|
case .flags(let flags):
|
|
currentFlags = flags
|
|
case .rfc822Size(let size):
|
|
currentSize = size
|
|
default:
|
|
break
|
|
}
|
|
case .streamingBegin, .streamingBytes, .streamingEnd:
|
|
break
|
|
case .finish:
|
|
if let uid = currentUID {
|
|
let envelope = buildFetchedEnvelope(
|
|
uid: uid,
|
|
envelope: currentEnvelope,
|
|
flags: currentFlags,
|
|
size: currentSize
|
|
)
|
|
envelopes.append(envelope)
|
|
}
|
|
}
|
|
default:
|
|
break
|
|
}
|
|
}
|
|
|
|
return envelopes
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Simplify buildFetchedEnvelope — remove bodyBuffer parameter**
|
|
|
|
Replace the method (bodies come from `prefetchBodies` now):
|
|
|
|
```swift
|
|
private func buildFetchedEnvelope(
|
|
uid: Int,
|
|
envelope: Envelope?,
|
|
flags: [Flag],
|
|
size: Int
|
|
) -> FetchedEnvelope {
|
|
let subject = envelope?.subject.flatMap { String(buffer: $0) }
|
|
let date = envelope?.date.map { String($0) } ?? ""
|
|
let messageId = envelope?.messageID.map { String($0) }
|
|
let inReplyTo = envelope?.inReplyTo.map { String($0) }
|
|
let from = envelope?.from.compactMap { extractEmailAddress($0) }.first
|
|
let to = envelope?.to.compactMap { extractEmailAddress($0) } ?? []
|
|
let cc = envelope?.cc.compactMap { extractEmailAddress($0) } ?? []
|
|
|
|
return FetchedEnvelope(
|
|
uid: uid,
|
|
messageId: messageId,
|
|
inReplyTo: inReplyTo,
|
|
references: nil,
|
|
subject: subject,
|
|
from: from,
|
|
to: to,
|
|
cc: cc,
|
|
date: date,
|
|
snippet: nil,
|
|
bodyText: nil,
|
|
bodyHtml: nil,
|
|
isRead: flags.contains(.seen),
|
|
isFlagged: flags.contains(.flagged),
|
|
size: size
|
|
)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: All tests pass. Existing sync tests use MockIMAPClient which doesn't change behavior. The `fullMessages` mock provides MIME bodies during prefetch.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Sources/IMAPClient/IMAPClient.swift
|
|
git commit -m "remove inline body fetch from fetchEnvelopes, bodies now come from MIME-aware prefetchBodies"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 14: Attachment download service
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SyncEngine/SyncCoordinator.swift`
|
|
|
|
The spec defines an attachment download flow: check cachePath → if cached open QuickLook → if not fetch via `fetchSection` → save to disk → update `cachePath`. This orchestration lives in SyncCoordinator since it coordinates IMAP + MailStore.
|
|
|
|
- [ ] **Step 1: Add downloadAttachment method**
|
|
|
|
Add to `SyncCoordinator.swift`:
|
|
|
|
```swift
|
|
// MARK: - Attachment Download
|
|
|
|
/// Download an attachment on demand, cache it to disk, update the DB record.
|
|
/// Returns the local file URL for preview.
|
|
public func downloadAttachment(
|
|
attachmentId: String,
|
|
messageUid: Int,
|
|
mailboxName: String
|
|
) async throws -> URL {
|
|
// Look up the attachment record
|
|
guard let record = try store.attachment(id: attachmentId) else {
|
|
throw AttachmentError.notFound
|
|
}
|
|
|
|
// Check if already cached
|
|
if let cachePath = record.cachePath {
|
|
let url = URL(fileURLWithPath: cachePath)
|
|
if FileManager.default.fileExists(atPath: cachePath) {
|
|
return url
|
|
}
|
|
}
|
|
|
|
guard let sectionPath = record.sectionPath else {
|
|
throw AttachmentError.noSectionPath
|
|
}
|
|
|
|
// Fetch the section data from IMAP
|
|
try await imapClient.connect()
|
|
let data = try await imapClient.fetchSection(uid: messageUid, mailbox: mailboxName, section: sectionPath)
|
|
try? await imapClient.disconnect()
|
|
|
|
// Build cache directory
|
|
let cacheDir = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first!
|
|
.appendingPathComponent("MagnumOpus")
|
|
.appendingPathComponent(accountConfig.id)
|
|
.appendingPathComponent("attachments")
|
|
try FileManager.default.createDirectory(at: cacheDir, withIntermediateDirectories: true)
|
|
|
|
// Determine file extension from filename
|
|
let ext = record.filename.flatMap { URL(fileURLWithPath: $0).pathExtension } ?? "bin"
|
|
let fileName = "\(attachmentId).\(ext)"
|
|
let fileURL = cacheDir.appendingPathComponent(fileName)
|
|
|
|
// Write data to disk
|
|
try data.write(to: fileURL)
|
|
|
|
// Update cache path in DB
|
|
try store.updateAttachmentCachePath(id: attachmentId, cachePath: fileURL.path)
|
|
|
|
return fileURL
|
|
}
|
|
|
|
public enum AttachmentError: Error {
|
|
case notFound
|
|
case noSectionPath
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Add `attachment(id:)` query to MailStore**
|
|
|
|
Add to `MailStore.swift` in the `// MARK: - Attachments` section:
|
|
|
|
```swift
|
|
public func attachment(id: String) throws -> AttachmentRecord? {
|
|
try dbWriter.read { db in
|
|
try AttachmentRecord.fetchOne(db, key: id)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Build and run tests**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build && swift test 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: Build and tests pass.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Sources/SyncEngine/SyncCoordinator.swift Sources/MailStore/MailStore.swift
|
|
git commit -m "add attachment download service: fetch section from IMAP, cache to disk"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 15: 25 MB per-file attachment size guard
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SMTPClient/MessageFormatter.swift`
|
|
- Modify: `Tests/SMTPClientTests/MessageFormatterTests.swift`
|
|
|
|
- [ ] **Step 1: Write failing test**
|
|
|
|
Add to `MessageFormatterTests.swift`:
|
|
|
|
```swift
|
|
@Test("formatMultipart throws for file exceeding 25 MB")
|
|
func attachmentSizeGuard() {
|
|
let message = OutgoingMessage(
|
|
from: EmailAddress(address: "alice@example.com"),
|
|
to: [EmailAddress(address: "bob@example.com")],
|
|
subject: "Big file",
|
|
bodyText: "See attached.",
|
|
messageId: "test@example.com"
|
|
)
|
|
// 26 MB attachment
|
|
let bigData = Data(repeating: 0xFF, count: 26 * 1024 * 1024)
|
|
let attachments: [(filename: String, mimeType: String, data: Data)] = [
|
|
(filename: "huge.bin", mimeType: "application/octet-stream", data: bigData),
|
|
]
|
|
|
|
#expect(throws: MessageFormatterError.self) {
|
|
try MessageFormatter.formatMultipart(message, attachments: attachments)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Add size guard and error type**
|
|
|
|
In `MessageFormatter.swift`, add at the bottom of the file:
|
|
|
|
```swift
|
|
public enum MessageFormatterError: Error {
|
|
case attachmentTooLarge(filename: String, size: Int)
|
|
}
|
|
```
|
|
|
|
Update `formatMultipart` signature to `throws` and add a size check at the start:
|
|
|
|
```swift
|
|
public static func formatMultipart(
|
|
_ message: OutgoingMessage,
|
|
attachments: [(filename: String, mimeType: String, data: Data)]
|
|
) throws -> String {
|
|
let maxSize = 25 * 1024 * 1024 // 25 MB
|
|
for attachment in attachments {
|
|
if attachment.data.count > maxSize {
|
|
throw MessageFormatterError.attachmentTooLarge(
|
|
filename: attachment.filename,
|
|
size: attachment.data.count
|
|
)
|
|
}
|
|
}
|
|
// ... rest of existing implementation unchanged
|
|
```
|
|
|
|
- [ ] **Step 3: Update non-throwing test calls to use `try`**
|
|
|
|
The existing multipart tests now need `try` since `formatMultipart` throws. Update test calls from:
|
|
```swift
|
|
let formatted = MessageFormatter.formatMultipart(message, attachments: attachments)
|
|
```
|
|
To:
|
|
```swift
|
|
let formatted = try MessageFormatter.formatMultipart(message, attachments: attachments)
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test --filter MessageFormatterTests 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: All tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Sources/SMTPClient/MessageFormatter.swift Tests/SMTPClientTests/MessageFormatterTests.swift
|
|
git commit -m "add 25 MB per-file attachment size guard to MessageFormatter"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 16: Update periodic sync interval & add SyncEngine dependency on MIMEParser to Package.swift project.yml
|
|
|
|
**Files:**
|
|
- Modify: `Sources/SyncEngine/SyncCoordinator.swift`
|
|
- Modify: `Apps/project.yml`
|
|
|
|
- [ ] **Step 1: Update default periodic sync interval**
|
|
|
|
In `SyncCoordinator.swift`, change the `startPeriodicSync` default interval:
|
|
|
|
```swift
|
|
public func startPeriodicSync(interval: Duration = .seconds(900)) {
|
|
```
|
|
|
|
(Changed from 300s/5min to 900s/15min — IDLE is the fast path now.)
|
|
|
|
- [ ] **Step 2: Add MIMEParser to project.yml**
|
|
|
|
In `Apps/project.yml`, add MIMEParser to the dependencies for both macOS and iOS targets. Look at how other products are listed and add:
|
|
|
|
```yaml
|
|
- package: MagnumOpusCore
|
|
product: MIMEParser
|
|
```
|
|
|
|
- [ ] **Step 3: Build and verify**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift build
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Sources/SyncEngine/SyncCoordinator.swift Apps/project.yml
|
|
git commit -m "bump periodic sync to 15min (IDLE is fast path), add MIMEParser to app targets"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 17: Final verification
|
|
|
|
- [ ] **Step 1: Run full test suite**
|
|
|
|
```bash
|
|
cd Packages/MagnumOpusCore
|
|
swift test 2>&1
|
|
```
|
|
|
|
Expected: All tests pass (original 111 + new tests).
|
|
|
|
- [ ] **Step 2: Build Xcode project**
|
|
|
|
```bash
|
|
cd /Users/felixfoertsch/Developer/MagnumOpus
|
|
xcodegen generate --spec Apps/project.yml --project Apps/
|
|
```
|
|
|
|
Then:
|
|
|
|
```bash
|
|
cd Apps
|
|
xcodebuild build -project MagnumOpus.xcodeproj -scheme MagnumOpus-macOS -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO 2>&1 | tail -10
|
|
```
|
|
|
|
Expected: Build succeeds.
|
|
|
|
- [ ] **Step 3: Commit any remaining changes**
|
|
|
|
```bash
|
|
cd /Users/felixfoertsch/Developer/MagnumOpus
|
|
git status
|
|
```
|
|
|
|
If clean, proceed. Otherwise commit remaining changes.
|
|
|
|
- [ ] **Step 4: Bump CalVer version**
|
|
|
|
Update the CalVer version in `Apps/project.yml` to today's date if not already done:
|
|
|
|
```yaml
|
|
MARKETING_VERSION: "2026.03.14"
|
|
```
|
|
|
|
(Already set from v0.4 — if this runs on a different day, update accordingly.)
|
|
|
|
- [ ] **Step 5: Final commit**
|
|
|
|
```bash
|
|
git add -A
|
|
git commit -m "v0.5: attachments (MIME parse, multipart send) + IMAP IDLE real-time mail"
|
|
```
|