Commit Graph
218 Commits
Author SHA1 Message Date
greatroarandGitHub 49488c0e71 all: Clean up fmt.Errorf usage (#8309) 2022-04-27 20:30:13 +02:00
greatroarandGitHub bd0acd04b1 lib/protocol: Use one mutex for rawConnect.awaiting and nextID (#8198)
Having a separate mutex for the three or four instructions needed to
fetch and increment nextID means the overhead exceeds the cost of this
operation.  nextID is now handled inside the critical section for
awaiting instead, while the more expensive channel creation has been
moved outside it.

This is mostly a simplification, though it may have minor performance
benefits in some situations. The single-threaded sender benchmark shows
no significant difference:

name               old speed      new speed      delta
RequestsRawTCP-8   55.3MB/s ± 7%  56.6MB/s ± 6%   ~     (p=0.190 n=10+10)
RequestsTLSoTCP-8  20.5MB/s ±20%  20.8MB/s ± 8%   ~     (p=0.604 n=10+9)
2022-02-28 09:13:30 +01:00
greatroarandGitHub a0fd619df3 lib/protocol: Require at least 3.125% savings from compression (#8133)
* lib/protocol: Require at least 3.125% savings from compression

The new lz4 library doesn't need its output buffer to be the maximum
size, unlike the old one (which would allocate if it weren't). It can
take a buffer that is of a smaller size and will report if compressed
data can fit inside the buffer (with a small chance of reporting a false
negative). Use that property to our advantage by requiring compressed
data to be at most n-n/32 = .96875*n bytes long for n input bytes.

* lib/protocol: Remove unused receivers

To make DeepSource happy.

* lib/protocol: Micro-optimize lz4Compress

Only write the length if compression was successful. This is a memory
write, so the compiler can't reorder it.

Only check the return value of lz4.CompressBlock.  Length-zero inputs
are always expanded by LZ4 compression (the library documents this),
so the check on len(src) isn't needed.
2022-01-24 19:36:58 +01:00
Jakob BorgandGitHub b6d1e16b4e lib/protocol: Switch to a newer lz4 package (#8122) 2022-01-17 18:52:43 +01:00
André ColombandGitHub 0cba3154f0 lib/model: Remove bogus fields from connections API endpoint (fixes #8103) (#8104)
* lib/model: Remove bogus fields from connections API endpoint.

Switch the returned data type for the /rest/system/connections element
"total" to use only the Statistics struct.  The other fields of the
ConnectionInfo struct are not populated and misleading.

* Lowercase JSON field names.

* lib/model: Get rid of ConnectionInfo.MarshalJSON().

It was missing the StartedAt field from the embedded Statistics
struct.  Just lowercasing the JSON attribute names can be done just as
easily with annotations.

* lib/model: Remove bogus startedAt field from totals.

Instead of using the Statistics type with one field empty, just switch
to a free-form map with the three needed fields.
2022-01-10 10:26:45 +01:00
greatroarandJakob Borg ae70046b49 lib/protocol: Remove unused sorting boilerplate 2021-11-27 15:35:07 +01:00
Jakob BorgandGitHub 4b750b6dc3 all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
greatroarandJakob Borg 8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroarandJakob Borg 46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon FreiandGitHub c94b797f00 lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Jakob Borg 92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
greatroarandJakob Borg eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Jakob BorgandGitHub d47745a86b all: Update build constraints to Go 1.17 style (#7894) 2021-08-17 10:10:41 +02:00
Simon FreiandGitHub 50aacdf1f0 lib/protocol: Ensure correct blocksize on enc. fileinfo (ref #7861) (#7870) 2021-08-04 23:12:01 +02:00
greatroarandGitHub bd363fe0b7 lib/protocol: Write uncompressible messages uncompressed (#7790) 2021-06-27 17:59:30 +02:00
Simon FreiandGitHub 857caf3637 lib/connections: Trigger dialer when connection gets closed (#7753) 2021-06-17 13:57:44 +02:00
Simon FreiandGitHub 5bdb6798a9 all: Regenerate proto (#7696) 2021-05-19 13:30:20 +02:00
greatroarandGitHub 3746c899b7 build: List go:generate tools in tools.go (#7599) 2021-05-08 12:52:06 +02:00
Simon FreiandGitHub 1a00ea7c6e lib: Prevent using protocol method with native path (fixes #7557) (#7563) 2021-04-11 15:29:43 +02:00
Simon FreiandGitHub 0dcd9794d4 lib/protocol: Deterministic encrypted version (fixes #7533) (#7538) 2021-03-31 08:59:15 +02:00
Simon FreiandGitHub 924b96856f lib: Handle adding enc folders on an existing conn (fixes #7509) (#7510) 2021-03-22 21:50:19 +01:00
Jakob BorgandGitHub 8ef504f745 all: Simplify some method calls (#7499)
strings.Replace(a, b, c, -1) -> strings.ReplaceAll(a, b, c)

(Go 1.12) and who knows what was up with that dialQueue.Sort() thing.
2021-03-17 23:12:26 +01:00
Simon FreiandGitHub d2d4fcc1df lib/protocol: Improve messages when an error occurs receiving (ref #7466) (#7470) 2021-03-15 19:14:09 +01:00
Simon FreiandGitHub f80ee472c2 lib/protocol: Set invalid flag on encrypted file infos (fixes #7466) (#7467) 2021-03-13 16:57:36 +01:00
Jakob BorgandGitHub 4d979a1ce9 all: Truncate some timestamps (fixes #7457) (#7459)
This truncates times meant for API consumption to second precision,
where fractions won't typically matter or add any value. Exception to
this is timestamps on logs and events, and of course I'm not touching
things like file metadata.

I'm not 100% certain this is an exhaustive change, but it's the things I
found by grepping and following the breadcrumbs from lib/api...

I also considered general-but-ugly solutions, like having the API
serializer itself do reflection magic or even regexps on returned
objects, but decided against it because aurgh...
2021-03-12 10:35:10 +01:00
greatroarandGitHub c00520281b lib/protocol: Optimize FileKey (#7440) 2021-03-07 18:44:21 +01:00
Simon FreiandGitHub 3d91f7c975 lib: Use counterfeiter to mock interfaces in tests (#7375) 2021-03-03 08:53:50 +01:00
greatroarandGitHub 56b5352f64 all: Use crypt/rand through its buffered version, but not in benchmarks (#7420) 2021-03-02 19:17:20 +01:00
greatroarandGitHub ffcaffa32f lib/protocol: Optimize encrypted filename handling + make it more strict (#7408) 2021-02-27 08:57:12 +01:00
Simon FreiandGitHub 0ffd80f380 lib/protocol: Alwasy return buffers to the pool (#7409) 2021-02-27 08:55:51 +01:00
Simon FreiandGitHub c0f353c0e8 lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Jakob BorgandGitHub 3b7a57d108 lib/protocol: Hide repeated data blocks in a given file (#7319) 2021-02-02 20:15:14 +01:00
Audrius ButkeviciusandGitHub a7d9268e4d lib/model: Make /browse endpoint return sane objects (#7306) 2021-02-01 09:27:34 +01:00
Jakob BorgandGitHub b13b15758d lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon FreiandGitHub c845e245a1 lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon FreiandGitHub 8fd6b1d428 lib/protocol: Handle slashified paths in IsEncryptedParent (fixes #7184) (#7187) 2020-12-09 18:16:14 +01:00
Simon FreiandGitHub bbb22c8c80 lib/protocol: Send Close message on read error (#7141) 2020-11-27 11:31:20 +01:00
Simon FreiandGitHub 5d2c83a7e9 lib/protocol: Fix OOR panic on recv-only folders (#7143) 2020-11-23 18:37:27 +01:00
Simon FreiandGitHub 31559e908b all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon FreiandGitHub f0f60ba2e7 lib/api: Add /rest/config endpoint (fixes #6540) (#7001) 2020-10-22 19:54:35 +02:00
Simon FreiandGitHub a20c6ca868 lib/model, lib/protocol: Send ClusterConfig on config change (fixes #7020) (#7018) 2020-10-02 11:49:51 +02:00
Audrius ButkeviciusandGitHub e027175446 all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Simon FreiandGitHub 1b1d38183d lib: Remove HelloResult type, same as Hello (#7015) 2020-09-29 12:17:38 +01:00
674fca3868 lib/db, lib/protocol: Never need empty-version entries (fixes #6961) (#6962)
Avoid havoc when discovering locally-deleted-upgraded files during repair / need calculation...

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-09-07 20:18:25 +02:00
Jakob Borg b7986801cd cmd/stindex: Print more hashes and versions 2020-09-04 09:10:33 +02:00
Audrius ButkeviciusandGitHub d507d932b8 all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Simon FreiandGitHub ebbe1abe28 lib/protocol: Include WeakHash in BlocksHash (#6799) 2020-06-29 09:33:07 +02:00
Simon FreiandGitHub d7fc7008af lib/protocol: Add BlocksHash to FileInfo string (#6802) 2020-06-27 07:44:33 +02:00
Simon FreiandGitHub 80ada1bb6c lib/protocol: Don't warn when dropping deleted invalid item (fixes #6795) (#6796) 2020-06-25 20:24:31 +02:00
greatroarandGitHub dc145bfad7 lib/protocol: Use sha256.Sum256 in NewDeviceID (#6775)
This is shorter, skips two allocations, makes the function inlineable
and is safer, since the compiler now check whether
DeviceIDLength == sha256.Size.
2020-06-21 20:38:06 +02:00