refactor(db): slightly improve insert performance (#10318)
This just removes an unnecessary foreign key constraint, where we
already do the garbage collection manually in the database service.
However, as part of getting here I tried a couple of other variants
along the way:
- Changing the order of the primary key from `(hash, blocklist_hash,
idx)` to `(blocklist_hash, idx, hash)` so that inserts would be
naturally ordered. However this requires a new index `on blocks (hash)`
so that we can still look up blocks by hash, and turns out to be
strictly worse than what we already have.
- Removing the primary key entirely and the `WITHOUT ROWID` to make it a
rowid table without any required order, and an index as above. This is
faster when the table is small, but becomes slower when it's large (due
to dual indexes I guess).
These are the benchmark results from current `main`, the second
alternative below ("Index(hash)") and this proposal that retains the
combined primary key ("combined"). Overall it ends up being about 65%
faster.
<img width="764" height="452" alt="Screenshot 2025-08-29 at 14 36 28"
src="https://github.com/user-attachments/assets/bff3f9d1-916a-485f-91b7-b54b477f5aac"
/>
Ref #10264
This commit is contained in:
@@ -114,11 +114,9 @@ func (s *folderDB) Update(device protocol.DeviceID, fs []protocol.FileInfo) erro
|
||||
if err != nil {
|
||||
return wrap(err, "marshal blocklist")
|
||||
}
|
||||
if _, err := insertBlockListStmt.Exec(f.BlocksHash, bs); err != nil {
|
||||
if res, err := insertBlockListStmt.Exec(f.BlocksHash, bs); err != nil {
|
||||
return wrap(err, "insert blocklist")
|
||||
}
|
||||
|
||||
if device == protocol.LocalDeviceID {
|
||||
} else if aff, _ := res.RowsAffected(); aff != 0 && device == protocol.LocalDeviceID {
|
||||
// Insert all blocks
|
||||
if err := s.insertBlocksLocked(txp, f.BlocksHash, f.Blocks); err != nil {
|
||||
return wrap(err, "insert blocks")
|
||||
|
||||
Reference in New Issue
Block a user