refactor(db): slightly improve insert performance (#10318)
This just removes an unnecessary foreign key constraint, where we
already do the garbage collection manually in the database service.
However, as part of getting here I tried a couple of other variants
along the way:
- Changing the order of the primary key from `(hash, blocklist_hash,
idx)` to `(blocklist_hash, idx, hash)` so that inserts would be
naturally ordered. However this requires a new index `on blocks (hash)`
so that we can still look up blocks by hash, and turns out to be
strictly worse than what we already have.
- Removing the primary key entirely and the `WITHOUT ROWID` to make it a
rowid table without any required order, and an index as above. This is
faster when the table is small, but becomes slower when it's large (due
to dual indexes I guess).
These are the benchmark results from current `main`, the second
alternative below ("Index(hash)") and this proposal that retains the
combined primary key ("combined"). Overall it ends up being about 65%
faster.
<img width="764" height="452" alt="Screenshot 2025-08-29 at 14 36 28"
src="https://github.com/user-attachments/assets/bff3f9d1-916a-485f-91b7-b54b477f5aac"
/>
Ref #10264
This commit is contained in:
@@ -21,14 +21,14 @@ CREATE TABLE IF NOT EXISTS blocklists (
|
||||
--
|
||||
-- For all local files we store the blocks individually for quick lookup. A
|
||||
-- given block can exist in multiple blocklists and at multiple offsets in a
|
||||
-- blocklist.
|
||||
-- blocklist. We eschew most indexes here as inserting millions of blocks is
|
||||
-- common and performance is critical.
|
||||
CREATE TABLE IF NOT EXISTS blocks (
|
||||
hash BLOB NOT NULL,
|
||||
blocklist_hash BLOB NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
offset INTEGER NOT NULL,
|
||||
size INTEGER NOT NULL,
|
||||
PRIMARY KEY (hash, blocklist_hash, idx),
|
||||
FOREIGN KEY(blocklist_hash) REFERENCES blocklists(blocklist_hash) ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED
|
||||
PRIMARY KEY(hash, blocklist_hash, idx)
|
||||
) STRICT, WITHOUT ROWID
|
||||
;
|
||||
|
||||
Reference in New Issue
Block a user