lib/db: Slightly improve indirection (ref #6372) (#6373)

I was working on indirecting version vectors, and that resulted in some
refactoring and improving the existing block indirection stuff. We may
or may not end up doing the version vector indirection, but I think
these changes are reasonable anyhow and will simplify the diff
significantly if we do go there. The main points are:

- A bunch of renaming to make the indirection and GC not about "blocks"
  but about "indirection".

- Adding a cutoff so that we don't actually indirect for small block
  lists. This gets us better performance when handling small files as it
  cuts out the indirection for quite small loss in space efficiency.

- Being paranoid and always recalculating the hash on put. This costs
  some CPU, but the consequences if a buggy or malicious implementation
  silently substituted the block list by lying about the hash would be bad.
This commit is contained in:
Jakob Borg
2020-02-27 11:19:21 +01:00
committed by GitHub
parent c4b9046eaa
commit 4f7a77597e
5 changed files with 94 additions and 75 deletions
+2 -2
View File
@@ -119,8 +119,8 @@ are mostly useful for developers. Use with care.
"h", "m" and "s" abbreviations for hours minutes and seconds.
Valid values are like "720h", "30s", etc.
STGCBLOCKSEVERY Set to a time interval to override the default database
block GC interval of 13 hours. Same format as the
STGCINDIRECTEVERY Set to a time interval to override the default database
indirection GC interval of 13 hours. Same format as the
STRECHECKDBEVERY variable.
GOMAXPROCS Set the maximum number of CPU cores to use. Defaults to all