1f8e6c55f6
lib/db: Refactor to use global list by version ( fixes #6372 ) ( #6638 )
...
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.
Co-authored-by: Jakob Borg <jakob@kastelo.net >
2020-05-30 09:50:23 +02:00
Simon Frei and GitHub
c3b5eba205
lib/model: Fix checking children when trying to delete a dir ( fixes #6646 ) ( #6647 )
2020-05-25 08:46:24 +02:00
Jakob Borg and GitHub
5ffa012410
lib/model: Don't crash when taking rename shortcut ( fixes #6654 ) ( #6657 )
...
If we fail to take the rename shortcut we may crash on a later loop,
because we do trickiness with the indexes but the original buckets[key]
in "range buckets[key]" isn't re-evaluated so i exceeds the max index.
2020-05-17 08:48:35 +01:00
Simon Frei and GitHub
299b9d8883
lib/model: Adjust remote-rename-test to timer-based versions ( fixes #6625 ) ( #6644 )
2020-05-14 00:31:05 +02:00
Audrius Butkevicius and GitHub
6201eebc98
lib/model: Add support for different puller block ordering ( #6587 )
...
* WIP
* Tests
* Header and format
* WIP
* Fix tests
* Goland you disappoint me
* Remove CC storage
* Update blockpullreorderer.go
2020-05-11 22:44:04 +01:00
Audrius Butkevicius and GitHub
decb967969
all: Reorder sequences for better rename detection ( #6574 )
2020-05-11 20:15:11 +02:00
Simon Frei and GitHub
2e3975e956
lib/model: Improve errors when deleting dirs ( fixes #6575 ) ( #6586 )
2020-05-01 11:11:38 +02:00
Audrius Butkevicius and GitHub
782bd08aad
lib/model: Add option to disable fsync ( #6588 )
...
* lib/model: Add option to disable fsync
* Fix test
* Dont open stuff for no reason
2020-05-01 08:36:46 +01:00
Jakob Borg and GitHub
6c73617974
lib/model: Use semaphore to limit concurrent folder writes ( fixes #6541 ) ( #6573 )
2020-04-27 00:13:18 +02:00
Simon Frei and GitHub
d3ed4de4ed
lib/model: Don't exit pullerRoutine on cancelled ctx ( fixes #6559 ) ( #6562 )
...
* lib/model: Don't exit pullerRoutine on cancelled ctx (fixes #6559 )
* actual fix
2020-04-21 18:55:14 +01:00
Simon Frei and GitHub
6bbd24de12
lib/model: Refactor folder health/error handling ( fixes #6557 ) ( #6558 )
2020-04-21 10:15:59 +02:00
greatroar and GitHub
2f26a95973
lib/db, lib/model: Code simplifications ( #6484 )
...
NamespacedKV.prefixedKey is still small enough to be inlined.
2020-03-31 14:32:24 +02:00
Jakob Borg and GitHub
20aaa5927b
lib/protocol: Use BlocksHash to compare block lists when available ( #6401 )
...
This is an optimization for faster equal checks on block lists.
2020-03-10 14:46:49 +01:00
Simon Frei and GitHub
f0e33d052a
lib: More contextification ( #6343 )
2020-02-24 21:57:15 +01:00
Jakob Borg and GitHub
55937b61ca
lib/model: Add global request limiter ( fixes #6302 ) ( #6303 )
...
This adds a new config with the simple and concise name
maxConcurrentIncomingRequestKiB. This limits how many bytes we have "in
the air" in the form of response data being read and processed.
After some testing I think that not having this limiter is seldom a
great idea and thus I propose a default value of 256 MiB for this new
setting.
I also refactored the folder IO limiter to be a model/folder attribute
instead of a package global.
2020-02-01 08:02:18 +01:00
Jakob Borg and GitHub
d91c4b010b
lib/config, lib/model: Limit concurrent pulls ( fixes #5914 ) ( #6290 )
...
Adds a new folder state "Waiting to Sync" in the same vein as the
existing "Waiting to Scan". This vastly improves performances in the
rare cases when there are lots and lots of folders operating.
2020-01-27 17:31:17 +01:00
Simon Frei and Jakob Borg
08f0e125ef
all: Transactionalize db.FileSet ( fixes #5952 ) ( #6239 )
2020-01-21 18:23:08 +01:00
Simon Frei and GitHub
08753ccabe
lib/model: Reset queue after all pulling is done ( fixes #5867 ) ( #6256 )
2020-01-08 12:21:22 +01:00
Simon Frei and GitHub
90d85fd0a2
lib: Replace done channel with contexts in and add names to util services ( #6166 )
2019-11-21 08:41:15 +01:00
Simon Frei and GitHub
0d14ee4142
lib/model: Don't info log repeat pull errors ( #6149 )
2019-11-19 09:56:53 +01:00
Simon Frei and Jakob Borg
ce72bee576
lib/model: Simplify pull error/retry logic ( fixes #6139 ) ( #6141 )
2019-11-11 15:50:28 +01:00
Simon Frei and Jakob Borg
bbdda059bd
lib/model: Check for symlinks before deleting during pull ( fixes #6090 ) ( #6100 )
2019-10-22 21:55:51 +02:00
Simon Frei and Jakob Borg
7b33294955
gui, lib/model: Add new state FolderPreparingSync ( fixes #6027 ) ( #6028 )
2019-10-16 09:08:54 +02:00
Audrius Butkevicius and GitHub
8fb576ed54
lib/model: Adjust blocks reported in usage reporting ( fixes #5995 ) ( #6037 )
...
* lib/model: Adjust blocks reported in usage reporting (fixes #5995 )
* Use variables, fix go.mod
2019-10-04 12:03:13 +01:00
Simon Frei and Jakob Borg
77a5980747
lib/model: Do free disk space check later on pull ( fixes #5948 ) ( #5949 )
2019-08-16 09:40:53 +02:00
Simon Frei and GitHub
b1c74860e8
all: Remove global events.Default (ref #4085 ) ( #5886 )
2019-08-15 16:29:37 +02:00
Simon Frei and Jakob Borg
05835ed81f
all: Remove potentially problematic errors from panics ( fixes #5839 ) ( #5912 )
2019-07-31 10:53:35 +02:00
Jakob Borg and GitHub
c1c976aa2b
lib/model: Don't panic on failed chmod-back on directory ( fixes #5836 ) ( #5896 )
...
* lib/model: Don't panic on failed chmod-back on directory (fixes #5836 )
This makes the "in writable dir"-wrapper log chmod-back errors instead
of panicking. To do that we need a logger so the function moved into the
model package which is also the only place it's used. The tests came
along.
(The test also exercised osutil.RenameOrCopy like some sort of
piggybacking. I removed that.)
2019-07-28 10:25:05 +02:00
Simon Frei and Jakob Borg
7a4c88d4e4
lib: Add mtime window when comparing files ( #5852 )
2019-07-23 21:48:53 +02:00
Simon Frei and Jakob Borg
20c8dbd9ed
lib/model: Fix integer conversion ( fixes #5837 ) ( #5851 )
2019-07-12 16:37:12 +02:00
Simon Frei and GitHub
ba056578ec
lib: Add util.Service as suture.Service template ( fixes #5801 ) ( #5806 )
2019-07-09 11:40:30 +02:00
Simon Frei and Jakob Borg
b58f6ca886
lib/model: Correct/unify check if item changed ( #5819 )
2019-06-29 07:45:41 +02:00
Simon Frei and Audrius Butkevicius
3c7e7e971d
lib/model: Make jobQueue.Jobs paginated ( fixes #5754 ) ( #5804 )
...
* lib/model: Make jobQueue.Jobs paginated (fixes #5754 )
* fix, no test yet
* add test
2019-06-27 19:25:38 +01:00
Simon Frei and GitHub
abd363e8bb
lib/model: Don't error on pulling deletion of invalid file ( fixes #5791 ) ( #5792 )
2019-06-14 08:48:14 +02:00
Audrius Butkevicius and Jakob Borg
e714df013f
lib/connections: Add QUIC protocol support ( fixes #5377 ) ( #5737 )
2019-05-29 09:56:40 +02:00
Simon Frei and GitHub
441ea109a1
lib/model: Refactor file deletions when pulling ( #5699 )
2019-05-17 18:29:54 +02:00
Simon Frei and Jakob Borg
b45d77b6be
lib/model: Fix regression deleting directories on pull (ref #5690 ) ( #5691 )
2019-05-06 20:55:26 +02:00
Simon Frei and Jakob Borg
79e67b7f79
lib/model: Remove individual pull errors on retry ( fixes #5659 ) ( #5690 )
2019-05-06 17:21:56 +02:00
Audrius Butkevicius and GitHub
0ca1f26ff8
lib/versioner: Restore for all versioners, cross-device support ( #5514 )
...
* lib/versioner: Restore for all versioners, cross-device support
Fixes #4631
Fixes #4586
Fixes #1634
Fixes #5338
Fixes #5419
2019-04-28 23:30:16 +01:00
Simon Frei and Audrius Butkevicius
110806842c
lib/model: Pass the old not new fileinfo to deleteItemOnDisk ( fixes #5654 ) ( #5655 )
2019-04-23 20:46:28 +01:00
Simon Frei and Audrius Butkevicius
926e9228ed
lib/scanner, lib/model: File -> item when logging error ( #5664 )
2019-04-21 16:19:59 +01:00
Simon Frei and GitHub
3bea59b0d9
lib/model: Refactor progressEmitter to de-/activate by config ( fixes #4613 ) ( #5623 )
2019-04-13 14:20:51 +02:00
Simon Frei and Jakob Borg
395e524e2d
lib/model: Update db on scan/pull in folder ( #5608 )
2019-04-07 13:29:17 +02:00
Simon Frei and Jakob Borg
8d1eff7e41
lib/model: Missing queue.Done and reset between pulls ( fixes #5332 ) ( #5626 )
2019-03-27 20:19:35 +01:00
Simon Frei and Audrius Butkevicius
0cff66fcbc
lib/model: Optimize puller for meta only changes ( #5622 )
2019-03-27 08:36:58 +00:00
Simon Frei and Jakob Borg
43a5be1c4b
lib/model: Send item finished even after deregistering ( fixes #5362 ) ( #5620 )
2019-03-26 21:31:33 +01:00
Simon Frei and Jakob Borg
d4e81fff8a
lib/model: Remove unnecessary arguments in pulling functions ( #5619 )
2019-03-25 12:59:22 +01:00
Simon Frei and Jakob Borg
e31a116e6e
lib/model: Pass correct file info to deleteItemOnDisk ( fixes #5616 ) ( #5617 )
2019-03-25 12:42:39 +01:00
Simon Frei and Audrius Butkevicius
289a02e994
lib/model: Integrate stat refs in folder ( #5596 )
2019-03-11 16:57:21 +00:00
Simon Frei and Jakob Borg
445637ebec
lib/model: Pass fset & ignores on folder creation ( #5592 )
2019-03-11 07:28:54 +01:00
Audrius ButkeviciusandGitHub
greatroarandGitHub
