Compare commits

...
32 Commits
Author SHA1 Message Date
Jakob Borg 2ea22b1850 gui, man: Update docs & translations
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3101
2016-05-17 12:02:44 +00:00
Jakob Borg 2c1323ece6 lib/connections: Un-deprecate relaysEnabled (fixes #3074)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3098
2016-05-17 00:05:38 +00:00
Audrius Butkevicius adb7fb43cb vendor: Update go-nat-pmp 2016-05-16 20:46:03 +01:00
AlexandJakob Borg d59fd9c22d lib/config: use correct ReleasesURL when upgrading from v0.13-beta
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3096
2016-05-14 22:03:07 +00:00
Jakob Borg 6f743f3138 Revert "lib/model: Emit LocalDiskUpdated events on detecting local changes"
This reverts commit 5a7fad0bcd.
2016-05-14 10:55:24 +02:00
Nate MorrisonandJakob Borg 5a7fad0bcd lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-14 08:37:07 +00:00
Jakob Borg 5d2414dfa9 lib/config: Bump config version to 14
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3092
2016-05-13 14:13:24 +00:00
Jakob Borg bef2425025 cmd/syncthing: Set User-Agent on upgrade checks
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3093
2016-05-13 14:11:59 +00:00
Jakob BorgandAudrius Butkevicius e8b4286c93 lib/config: Change upgrade check URL (fixes #3086)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3089
2016-05-13 09:17:10 +00:00
Jakob BorgandAudrius Butkevicius 2e9bf0b67c lib/upgrade: Increase size limits, send version header
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3088
2016-05-13 09:01:31 +00:00
Lars K.W. GohlkeandAudrius Butkevicius 935c273c8f cleanup: removed deadcode in connection/tcp_listen.go
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3084
2016-05-12 20:43:11 +00:00
Jakob BorgandAudrius Butkevicius b993b41847 lib/config: Minor attribute updates
As discussed in
https://github.com/syncthing/docs/pull/169

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3082
2016-05-12 08:23:18 +00:00
Jakob BorgandAudrius Butkevicius 1be40cc4fa lib/ignore: Revert comma handling, upgrade globbing package
This was fixed upstream due to our ticket, so we no longer need the
manual handling of commas. Keep the tests and better debug output around
though.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3081
2016-05-12 07:11:16 +00:00
Lars K.W. GohlkeandJakob Borg d628b731d1 build: Remove unused code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3079
2016-05-11 06:21:30 +00:00
Jakob BorgandAudrius Butkevicius 21e116aa45 lib/scanner: Refactor scanner.Walk API
The old usage pattern was to create a Walker with a bunch of attributes,
then call Walk() on it and nothing else. This extracts the attributes
into a Config struct and exposes a Walk(cfg Config) method instead, as
there was no reason to expose the state-holding walker type.

Also creates a few no-op implementations of the necessary interfaces
so that we can skip nil checks and simiplify things here and there.

Definitely look at this diff without whitespace.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3060
2016-05-09 18:25:39 +00:00
Jakob BorgandAudrius Butkevicius d77d8ff803 lib/connections: Don't look at devices that are already optimally connected
Just an optimization. Required exposing the priority from the factory,
so made that an interface with an extra method instead of just a func
type.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3071
2016-05-09 15:33:25 +00:00
Jakob BorgandAudrius Butkevicius 31f64186ae lib/connections: More fine grained locking (fixes #3066)
This fixes the deadlock by reducing where we hold the various locks. To
start with it splits up the existing "mut" into a "listenersMut" and a
"curConMut" as these are the two things being protected and I can see no
relation between them that requires a shared lock. It also moves all
model calls outside of the lock, as I see no reason to hold the lock
while calling the model (and it's risky, as proven).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3069
2016-05-09 15:03:12 +00:00
Jakob BorgandAudrius Butkevicius 1a703efa78 lib/model: Fix accounting error in rescan with multiple subs (fixes #3028)
When doing prefix scans in the database, "foo" should not be considered
a prefix of "foo2". Instead, it should match "foo" exactly and also
strings with the prefix "foo/". This is more restrictive than what the
standard leveldb prefix scan does so we add some code to enforce it.

Also exposes the initialScanCompleted on the rwfolder for testing, and
change it to be a channel (so we can wait for it from another
goroutine). Otherwise we can't be sure when the initial scan has
completed, and we need to wait for that or it might pick up changes
we're doing at an unexpected time.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3067
2016-05-09 12:56:21 +00:00
Jakob BorgandAudrius Butkevicius 8b7b0a03eb lib/config: Don't require restart when adding folders/devices or changing listen address
The VersioningConfig change is because it defaults to nil but gets
deserialized to map[string]string{}. Now prepare() enforces a single
representation of the empty map.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3065
2016-05-09 11:30:19 +00:00
Jakob BorgandAudrius Butkevicius 0761d804a4 cmd/syncthing: Use random folder ID for default folder, limit random charset
This uses the same charset as the Javascript code, excluding confusing
characters like 0, O, I, 1, l etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3064
2016-05-09 09:43:40 +00:00
Jakob BorgandAudrius Butkevicius 3ad42d9279 lib/util: Should seed random number generator on startup
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3063
2016-05-09 09:36:42 +00:00
klemensandJakob Borg bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg 10fe23b8f2 script: Don't verify authors on commits tagged 'Skip-check: authors'
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3057
2016-05-08 10:47:57 +00:00
Jakob BorgandAudrius Butkevicius 39899e40bf cmd/syncthing: Use ReadAll + json.Unmarshal in places were we care about consuming the reader
Because json.NewDecoder(r).Decode(&v) doesn't necessarily consume all
data on the reader, that means an HTTP connection can't be reused. We
don't do a lot of HTTP traffic where we read JSON responses, but the
discovery is one such place. The other two are for POSTs from the GUI,
where it's not exactly critical but still nice if the connection still
can be keep-alive'd after the request as well.

Also ensure that we call req.Body.Close() for clarity, even though this
should by all accounts not really be necessary.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3050
2016-05-06 22:01:56 +00:00
Jakob BorgandAudrius Butkevicius 5d337bb24f lib/ignore: Handle bare commas in ignore patterns (fixes #3042)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3048
2016-05-06 15:45:11 +00:00
Jakob BorgandAudrius Butkevicius dd5909568f lib/upgrade: Don't attempt processing files larger than expected max binary size (ref #3045)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3047
2016-05-06 14:14:19 +00:00
Jakob BorgandAudrius Butkevicius 38166e976f lib/upgrade: Enforce limits on download archives (fixes #3045)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3046
2016-05-06 13:58:34 +00:00
Jakob BorgandAudrius Butkevicius d6a7ffe0d4 lib/upgrade: Auto upgrade signature should cover version & arch (fixes #3044)
New signature is the HMAC of archive name (which includes the release
version and architecture) plus the contents of the binary. This is
expected in a new file "release.sig" which may be present in a
subdirectory. The new release tools put this in [.]metadata/release.sig.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3043
2016-05-06 13:30:35 +00:00
Jakob BorgandAudrius Butkevicius 2ebc6996a2 cmd/stsigtool: Sign stdin when not given a file to sign, or when given "-"
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3041
2016-05-05 19:05:45 +00:00
Jakob BorgandAudrius Butkevicius 2e840134d2 lib/protocol: Add Request benchmarks over raw and TLS encrypted TCP channels
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3040
2016-05-04 23:07:07 +00:00
Jakob Borg 66e1be33cf lib/protocol: Delete erroneously checked in test binary
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3039
2016-05-04 22:09:07 +00:00
Jakob BorgandAudrius Butkevicius 591959261c gui: Fix comparison operator in expression (ref #3035)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3036
2016-05-04 20:30:18 +00:00
68 changed files with 1321 additions and 502 deletions
-17
View File
@@ -674,13 +674,6 @@ func buildHost() string {
return h
}
func buildEnvironment() string {
if v := os.Getenv("ENVIRONMENT"); len(v) > 0 {
return v
}
return "default"
}
func buildArch() string {
os := goos
if os == "darwin" {
@@ -693,16 +686,6 @@ func archiveName(target target) string {
return fmt.Sprintf("%s-%s-%s", target.name, buildArch(), version)
}
func run(cmd string, args ...string) []byte {
bs, err := runError(cmd, args...)
if err != nil {
log.Println(cmd, strings.Join(args, " "))
log.Println(string(bs))
log.Fatal(err)
}
return bytes.TrimSpace(bs)
}
func runError(cmd string, args ...string) ([]byte, error) {
ecmd := exec.Command(cmd, args...)
bs, err := ecmd.CombinedOutput()
+2 -2
View File
@@ -29,7 +29,7 @@ var (
var (
// Static prefix that we use when generating fake device IDs, so that we
// can recognize them ourselves. Also makes the device ID start with
// "STPROBE-" which is humanly recognizeable.
// "STPROBE-" which is humanly recognizable.
randomPrefix = []byte{148, 223, 23, 4, 148}
// Our random, fake, device ID that we use when sending announcements.
@@ -117,7 +117,7 @@ func addrStrs(dev discover.Device) []string {
return ss
}
// returns a random but recognizeable device ID
// returns a random but recognizable device ID
func randomDeviceID() []byte {
var id [32]byte
copy(id[:], randomPrefix)
+13 -6
View File
@@ -8,6 +8,7 @@ package main
import (
"flag"
"io"
"io/ioutil"
"log"
"os"
@@ -31,7 +32,7 @@ Where command is one of:
gen
- generate a new key pair
sign <privkeyfile> <datafile>
sign <privkeyfile> [datafile]
- sign a file
verify <signaturefile> <datafile>
@@ -72,13 +73,19 @@ func sign(keyname, dataname string) {
log.Fatal(err)
}
fd, err := os.Open(dataname)
if err != nil {
log.Fatal(err)
var input io.Reader
if dataname == "-" || dataname == "" {
input = os.Stdin
} else {
fd, err := os.Open(dataname)
if err != nil {
log.Fatal(err)
}
defer fd.Close()
input = fd
}
defer fd.Close()
sig, err := signature.Sign(privkey, fd)
sig, err := signature.Sign(privkey, input)
if err != nil {
log.Fatal(err)
}
+9 -3
View File
@@ -718,6 +718,7 @@ func (s *apiService) postSystemConfig(w http.ResponseWriter, r *http.Request) {
defer s.systemConfigMut.Unlock()
to, err := config.ReadJSON(r.Body, myID)
r.Body.Close()
if err != nil {
l.Warnln("decoding posted config:", err)
http.Error(w, err.Error(), http.StatusBadRequest)
@@ -940,10 +941,15 @@ func (s *apiService) getDBIgnores(w http.ResponseWriter, r *http.Request) {
func (s *apiService) postDBIgnores(w http.ResponseWriter, r *http.Request) {
qs := r.URL.Query()
var data map[string][]string
err := json.NewDecoder(r.Body).Decode(&data)
bs, err := ioutil.ReadAll(r.Body)
r.Body.Close()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
var data map[string][]string
err = json.Unmarshal(bs, &data)
if err != nil {
http.Error(w, err.Error(), 500)
return
@@ -1205,7 +1211,7 @@ func (s embeddedStatic) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Check for a compiled in asset for the current theme.
bs, ok := s.assets[theme+"/"+file]
if !ok {
// Check for an overriden default asset.
// Check for an overridden default asset.
if s.assetDir != "" {
p := filepath.Join(s.assetDir, config.DefaultTheme, filepath.FromSlash(file))
if _, err := os.Stat(p); err == nil {
+2 -2
View File
@@ -137,13 +137,13 @@ func TestAssetsDir(t *testing.T) {
// assetsdir/foo/a exists, overrides compiled in
expectURLToContain(t, s.URL+"/a", "overridden-foo")
// foo/b is compiled in, default/b is overriden, return compiled in
// foo/b is compiled in, default/b is overridden, return compiled in
expectURLToContain(t, s.URL+"/b", "foo")
// only exists as compiled in default/c so use that
expectURLToContain(t, s.URL+"/c", "default")
// only exists as overriden default/d so use that
// only exists as overridden default/d so use that
expectURLToContain(t, s.URL+"/d", "overridden-default")
}
+3 -2
View File
@@ -946,8 +946,9 @@ func defaultConfig(myName string) config.Configuration {
if !noDefaultFolder {
l.Infoln("Default folder created and/or linked to new config")
defaultFolder = config.NewFolderConfiguration("default", locations[locDefFolder])
folderID := util.RandomString(5) + "-" + util.RandomString(5)
defaultFolder = config.NewFolderConfiguration(folderID, locations[locDefFolder])
defaultFolder.Label = "Default Folder (" + folderID + ")"
defaultFolder.RescanIntervalS = 60
defaultFolder.MinDiskFreePct = 1
defaultFolder.Devices = []config.FolderDeviceConfiguration{{DeviceID: myID}}
+1 -1
View File
@@ -195,7 +195,7 @@ code.ng-binding{
}
/* progess bars */
/* progress bars */
.progress-bar {
background-color: #217dbb !important;
}
+2 -2
View File
@@ -152,7 +152,7 @@
"Reused": "Réutilisé",
"Save": "Sauver",
"Scan Time Remaining": "Intervalle entre chaque analyse",
"Scanning": "En cours d'analyse",
"Scanning": "Analyse en cours",
"Select the devices to share this folder with.": "Sélectionner les machines avec qui partager ce dossier.",
"Select the folders to share with this device.": "Sélectionner les dossiers à partager avec cette machine.",
"Settings": "Configuration",
@@ -179,7 +179,7 @@
"Stopped": "Arrêté",
"Support": "Aide",
"Sync Protocol Listen Addresses": "Adresse d'écoute du protocole de synchronisation",
"Syncing": "En cours de synchronisation",
"Syncing": "Synchronisation en cours",
"Syncthing has been shut down.": "Syncthing a été éteint.",
"Syncthing includes the following software or portions thereof:": "Syncthing intègre les logiciels suivants (ou des éléments provenant de ces logiciels) :",
"Syncthing is restarting.": "Syncthing est cours de redémarrage.",
+1 -1
View File
@@ -55,7 +55,7 @@
"Edit Device": "Apparaat bewurkje",
"Edit Folder": "Map bewurkje",
"Editing": "Bewurkjen",
"Enable NAT traversal": "Enable NAT traversal",
"Enable NAT traversal": "NAT-trochkruse ynskeakelje",
"Enable Relaying": "Trochjaan tastean",
"Enable UPnP": "UPnP oansette",
"Enter comma separated (\"tcp://ip:port\", \"tcp://host:port\") addresses or \"dynamic\" to perform automatic discovery of the address.": "Fier troch komma's skieden (\"tcp://ip:port\", \"tcp://host:port\") adressen yn of \"dynamic\" om automatyske ûntdekking fan it adres út te fieren.",
+1 -1
View File
@@ -366,7 +366,7 @@
</table>
</div>
<div class="panel-footer">
<button type="button" class="btn btn-sm btn-danger pull-left" ng-click="override(folder.id)" ng-if="folderStatus(folder) == 'outofsync' && folder.type = 'readonly'">
<button type="button" class="btn btn-sm btn-danger pull-left" ng-click="override(folder.id)" ng-if="folderStatus(folder) == 'outofsync' && folder.type == 'readonly'">
<span class="fa fa-arrow-circle-up"></span>&nbsp;<span translate>Override Changes</span>
</button>
<span class="pull-right">
@@ -56,11 +56,24 @@
</div>
</div>
<div class="form-group">
<div class="checkbox">
<label>
<input id="GlobalAnnEnabled" type="checkbox" ng-model="tmpOptions.globalAnnounceEnabled"> <span translate>Global Discovery</span>
</label>
<div class="row">
<div class="col-md-6">
<div class="form-group">
<div class="checkbox">
<label>
<input id="GlobalAnnEnabled" type="checkbox" ng-model="tmpOptions.globalAnnounceEnabled"> <span translate>Global Discovery</span>
</label>
</div>
</div>
</div>
<div class="col-md-6">
<div class="form-group">
<div class="checkbox">
<label>
<input id="RelaysEnabled" type="checkbox" ng-model="tmpOptions.relaysEnabled"> <span translate>Enable Relaying</span>
</label>
</div>
</div>
</div>
</div>
+71 -24
View File
@@ -11,6 +11,7 @@ import (
"encoding/json"
"encoding/xml"
"io"
"io/ioutil"
"net/url"
"os"
"sort"
@@ -22,15 +23,15 @@ import (
const (
OldestHandledVersion = 10
CurrentVersion = 13
CurrentVersion = 14
MaxRescanIntervalS = 365 * 24 * 60 * 60
)
var (
// DefaultListenAddresses should be substituted when the configuration
// contains <listenAddress>default</listenAddress>. This is
// done by the "consumer" of the configuration, as we don't want these
// saved to the config.
// contains <listenAddress>default</listenAddress>. This is done by the
// "consumer" of the configuration as we don't want these saved to the
// config.
DefaultListenAddresses = []string{
"tcp://0.0.0.0:22000",
"dynamic+https://relays.syncthing.net/endpoint",
@@ -92,7 +93,12 @@ func ReadJSON(r io.Reader, myID protocol.DeviceID) (Configuration, error) {
util.SetDefaults(&cfg.Options)
util.SetDefaults(&cfg.GUI)
err := json.NewDecoder(r).Decode(&cfg)
bs, err := ioutil.ReadAll(r)
if err != nil {
return cfg, err
}
err = json.Unmarshal(bs, &cfg)
cfg.OriginalVersion = cfg.Version
cfg.prepare(myID)
@@ -192,6 +198,9 @@ func (cfg *Configuration) prepare(myID protocol.DeviceID) {
if cfg.Version == 12 {
convertV12V13(cfg)
}
if cfg.Version == 13 {
convertV13V14(cfg)
}
// Build a list of available devices
existingDevices := make(map[protocol.DeviceID]bool)
@@ -245,31 +254,57 @@ func (cfg *Configuration) prepare(myID protocol.DeviceID) {
}
}
func convertV12V13(cfg *Configuration) {
func convertV13V14(cfg *Configuration) {
// Not using the ignore cache is the new default. Disable it on existing
// configurations.
cfg.Options.CacheIgnoredFiles = false
cfg.Options.NATEnabled = cfg.Options.DeprecatedUPnPEnabled
cfg.Options.NATLeaseM = cfg.Options.DeprecatedUPnPLeaseM
cfg.Options.NATRenewalM = cfg.Options.DeprecatedUPnPRenewalM
cfg.Options.NATTimeoutS = cfg.Options.DeprecatedUPnPTimeoutS
if cfg.Options.DeprecatedRelaysEnabled {
cfg.Options.ListenAddresses = append(cfg.Options.ListenAddresses, cfg.Options.DeprecatedRelayServers...)
// Replace our two fairly long addresses with 'default' if both exist.
var newAddresses []string
for _, addr := range cfg.Options.ListenAddresses {
if addr != "tcp://0.0.0.0:22000" && addr != "dynamic+https://relays.syncthing.net/endpoint" {
newAddresses = append(newAddresses, addr)
}
}
if len(newAddresses)+2 == len(cfg.Options.ListenAddresses) {
cfg.Options.ListenAddresses = append([]string{"default"}, newAddresses...)
// Migrate UPnP -> NAT options
cfg.Options.NATEnabled = cfg.Options.DeprecatedUPnPEnabled
cfg.Options.DeprecatedUPnPEnabled = false
cfg.Options.NATLeaseM = cfg.Options.DeprecatedUPnPLeaseM
cfg.Options.DeprecatedUPnPLeaseM = 0
cfg.Options.NATRenewalM = cfg.Options.DeprecatedUPnPRenewalM
cfg.Options.DeprecatedUPnPRenewalM = 0
cfg.Options.NATTimeoutS = cfg.Options.DeprecatedUPnPTimeoutS
cfg.Options.DeprecatedUPnPTimeoutS = 0
// Replace the default listen address "tcp://0.0.0.0:22000" with the
// string "default", but only if we also have the default relay pool
// among the relay servers as this is implied by the new "default"
// entry.
hasDefault := false
for _, raddr := range cfg.Options.DeprecatedRelayServers {
if raddr == "dynamic+https://relays.syncthing.net/endpoint" {
for i, addr := range cfg.Options.ListenAddresses {
if addr == "tcp://0.0.0.0:22000" {
cfg.Options.ListenAddresses[i] = "default"
hasDefault = true
break
}
}
break
}
}
cfg.Options.DeprecatedRelaysEnabled = false
// Copy relay addresses into listen addresses.
for _, addr := range cfg.Options.DeprecatedRelayServers {
if hasDefault && addr == "dynamic+https://relays.syncthing.net/endpoint" {
// Skip the default relay address if we already have the
// "default" entry in the list.
continue
}
if addr == "" {
continue
}
cfg.Options.ListenAddresses = append(cfg.Options.ListenAddresses, addr)
}
cfg.Options.DeprecatedRelayServers = nil
// For consistency
sort.Strings(cfg.Options.ListenAddresses)
var newAddrs []string
for _, addr := range cfg.Options.GlobalAnnServers {
if addr != "default" {
@@ -285,8 +320,6 @@ func convertV12V13(cfg *Configuration) {
}
cfg.Options.GlobalAnnServers = newAddrs
cfg.Version = 13
for i, fcfg := range cfg.Folders {
if fcfg.DeprecatedReadOnly {
cfg.Folders[i].Type = FolderTypeReadOnly
@@ -295,6 +328,20 @@ func convertV12V13(cfg *Configuration) {
}
cfg.Folders[i].DeprecatedReadOnly = false
}
// v0.13-beta already had config version 13 but did not get the new URL
if cfg.Options.ReleasesURL == "https://api.github.com/repos/syncthing/syncthing/releases?per_page=30" {
cfg.Options.ReleasesURL = "https://upgrades.syncthing.net/meta.json"
}
cfg.Version = 14
}
func convertV12V13(cfg *Configuration) {
if cfg.Options.ReleasesURL == "https://api.github.com/repos/syncthing/syncthing/releases?per_page=30" {
cfg.Options.ReleasesURL = "https://upgrades.syncthing.net/meta.json"
}
cfg.Version = 13
}
func convertV11V12(cfg *Configuration) {
+65 -3
View File
@@ -12,7 +12,9 @@ import (
"fmt"
"os"
"path/filepath"
"reflect"
"runtime"
"sort"
"strings"
"testing"
@@ -40,6 +42,7 @@ func TestDefaultValues(t *testing.T) {
MaxSendKbps: 0,
MaxRecvKbps: 0,
ReconnectIntervalS: 60,
RelaysEnabled: true,
RelayReconnectIntervalM: 10,
StartBrowser: true,
NATEnabled: true,
@@ -57,9 +60,9 @@ func TestDefaultValues(t *testing.T) {
URURL: "https://data.syncthing.net/newdata",
URInitialDelayS: 1800,
URPostInsecurely: false,
ReleasesURL: "https://api.github.com/repos/syncthing/syncthing/releases?per_page=30",
ReleasesURL: "https://upgrades.syncthing.net/meta.json",
AlwaysLocalNets: []string{},
OverwriteNames: false,
OverwriteRemoteDevNames: false,
TempIndexMinBlocks: 10,
}
@@ -169,6 +172,7 @@ func TestOverriddenValues(t *testing.T) {
MaxSendKbps: 1234,
MaxRecvKbps: 2341,
ReconnectIntervalS: 6000,
RelaysEnabled: false,
RelayReconnectIntervalM: 20,
StartBrowser: false,
NATEnabled: false,
@@ -188,7 +192,7 @@ func TestOverriddenValues(t *testing.T) {
URPostInsecurely: true,
ReleasesURL: "https://localhost/releases",
AlwaysLocalNets: []string{},
OverwriteNames: true,
OverwriteRemoteDevNames: true,
TempIndexMinBlocks: 100,
}
@@ -616,3 +620,61 @@ func TestRemoveDuplicateDevicesFolders(t *testing.T) {
t.Errorf("Incorrect number of folder devices, %d != 2", l)
}
}
func TestV14ListenAddressesMigration(t *testing.T) {
tcs := [][3][]string{
// Default listen plus default relays is now "default"
{
{"tcp://0.0.0.0:22000"},
{"dynamic+https://relays.syncthing.net/endpoint"},
{"default"},
},
// Default listen address without any relay addresses gets converted
// to just the listen address. It's easier this way, and frankly the
// user has gone to some trouble to get the empty string in the
// config to start with...
{
{"tcp://0.0.0.0:22000"}, // old listen addrs
{""}, // old relay addrs
{"tcp://0.0.0.0:22000"}, // new listen addrs
},
// Default listen plus non-default relays gets copied verbatim
{
{"tcp://0.0.0.0:22000"},
{"dynamic+https://other.example.com"},
{"tcp://0.0.0.0:22000", "dynamic+https://other.example.com"},
},
// Non-default listen plus default relays gets copied verbatim
{
{"tcp://1.2.3.4:22000"},
{"dynamic+https://relays.syncthing.net/endpoint"},
{"tcp://1.2.3.4:22000", "dynamic+https://relays.syncthing.net/endpoint"},
},
// Default stuff gets sucked into "default", the rest gets copied
{
{"tcp://0.0.0.0:22000", "tcp://1.2.3.4:22000"},
{"dynamic+https://relays.syncthing.net/endpoint", "relay://other.example.com"},
{"default", "tcp://1.2.3.4:22000", "relay://other.example.com"},
},
}
for _, tc := range tcs {
cfg := Configuration{
Version: 13,
Options: OptionsConfiguration{
ListenAddresses: tc[0],
DeprecatedRelayServers: tc[1],
},
}
convertV13V14(&cfg)
if cfg.Version != 14 {
t.Error("Configuration was not converted")
}
sort.Strings(tc[2])
if !reflect.DeepEqual(cfg.Options.ListenAddresses, tc[2]) {
t.Errorf("Migration error; actual %#v != expected %#v", cfg.Options.ListenAddresses, tc[2])
}
}
}
+5 -1
View File
@@ -42,7 +42,7 @@ type FolderConfiguration struct {
Invalid string `xml:"-" json:"invalid"` // Set at runtime when there is an error, not saved
cachedPath string
DeprecatedReadOnly bool `xml:"ro,attr" json:"-"`
DeprecatedReadOnly bool `xml:"ro,attr,omitempty" json:"-"`
}
type FolderDeviceConfiguration struct {
@@ -138,6 +138,10 @@ func (f *FolderConfiguration) prepare() {
} else if f.RescanIntervalS < 0 {
f.RescanIntervalS = 0
}
if f.Versioning.Params == nil {
f.Versioning.Params = make(map[string]string)
}
}
func (f *FolderConfiguration) cleanedPath() string {
+8 -8
View File
@@ -16,6 +16,7 @@ type OptionsConfiguration struct {
MaxSendKbps int `xml:"maxSendKbps" json:"maxSendKbps"`
MaxRecvKbps int `xml:"maxRecvKbps" json:"maxRecvKbps"`
ReconnectIntervalS int `xml:"reconnectionIntervalS" json:"reconnectionIntervalS" default:"60"`
RelaysEnabled bool `xml:"relaysEnabled" json:"relaysEnabled" default:"true"`
RelayReconnectIntervalM int `xml:"relayReconnectIntervalM" json:"relayReconnectIntervalM" default:"10"`
StartBrowser bool `xml:"startBrowser" json:"startBrowser" default:"true"`
NATEnabled bool `xml:"natEnabled" json:"natEnabled" default:"true"`
@@ -35,17 +36,16 @@ type OptionsConfiguration struct {
SymlinksEnabled bool `xml:"symlinksEnabled" json:"symlinksEnabled" default:"true"`
LimitBandwidthInLan bool `xml:"limitBandwidthInLan" json:"limitBandwidthInLan" default:"false"`
MinHomeDiskFreePct float64 `xml:"minHomeDiskFreePct" json:"minHomeDiskFreePct" default:"1"`
ReleasesURL string `xml:"releasesURL" json:"releasesURL" default:"https://api.github.com/repos/syncthing/syncthing/releases?per_page=30"`
ReleasesURL string `xml:"releasesURL" json:"releasesURL" default:"https://upgrades.syncthing.net/meta.json"`
AlwaysLocalNets []string `xml:"alwaysLocalNet" json:"alwaysLocalNets"`
OverwriteNames bool `xml:"overwriteNames" json:"overwriteNames" default:"false"`
OverwriteRemoteDevNames bool `xml:"overwriteRemoteDeviceNamesOnConnect" json:"overwriteRemoteDeviceNamesOnConnect" default:"false"`
TempIndexMinBlocks int `xml:"tempIndexMinBlocks" json:"tempIndexMinBlocks" default:"10"`
DeprecatedUPnPEnabled bool `xml:"upnpEnabled" json:"-"`
DeprecatedUPnPLeaseM int `xml:"upnpLeaseMinutes" json:"-"`
DeprecatedUPnPRenewalM int `xml:"upnpRenewalMinutes" json:"-"`
DeprecatedUPnPTimeoutS int `xml:"upnpTimeoutSeconds" json:"-"`
DeprecatedRelaysEnabled bool `xml:"relaysEnabled" json:"-"`
DeprecatedRelayServers []string `xml:"relayServer" json:"-"`
DeprecatedUPnPEnabled bool `xml:"upnpEnabled,omitempty" json:"-"`
DeprecatedUPnPLeaseM int `xml:"upnpLeaseMinutes,omitempty" json:"-"`
DeprecatedUPnPRenewalM int `xml:"upnpRenewalMinutes,omitempty" json:"-"`
DeprecatedUPnPTimeoutS int `xml:"upnpTimeoutSeconds,omitempty" json:"-"`
DeprecatedRelayServers []string `xml:"relayServer,omitempty" json:"-"`
}
func (orig OptionsConfiguration) Copy() OptionsConfiguration {
+2 -2
View File
@@ -1,4 +1,4 @@
<configuration version="13">
<configuration version="14">
<options>
<listenAddress>tcp://:23000</listenAddress>
<allowDelete>false</allowDelete>
@@ -32,7 +32,7 @@
<urInitialDelayS>800</urInitialDelayS>
<urPostInsecurely>true</urPostInsecurely>
<releasesURL>https://localhost/releases</releasesURL>
<overwriteNames>true</overwriteNames>
<overwriteRemoteDeviceNamesOnConnect>true</overwriteRemoteDeviceNamesOnConnect>
<tempIndexMinBlocks>100</tempIndexMinBlocks>
</options>
</configuration>
+1 -1
View File
@@ -1,5 +1,5 @@
<configuration version="13">
<folder id="test" path="testdata" type="readonly" ignorePerms="false" rescanIntervalS="600" autoNormalize="true">
<folder id="test" path="testdata" ro="true" ignorePerms="false" rescanIntervalS="600" autoNormalize="true">
<device id="AIR6LPZ-7K4PTTV-UXQSMUU-CPQ5YWH-OEDFIIQ-JUG777G-2YQXXR5-YD6AWQR"></device>
<device id="P56IOI7-MZJNU2Y-IQGDREY-DM2MGTI-MGL3BXN-PQ6W5BM-TBBZ4TJ-XZWICQ2"></device>
<minDiskFreePct>1</minDiskFreePct>
+14
View File
@@ -0,0 +1,14 @@
<configuration version="14">
<folder id="test" path="testdata" type="readonly" ignorePerms="false" rescanIntervalS="600" autoNormalize="true">
<device id="AIR6LPZ-7K4PTTV-UXQSMUU-CPQ5YWH-OEDFIIQ-JUG777G-2YQXXR5-YD6AWQR"></device>
<device id="P56IOI7-MZJNU2Y-IQGDREY-DM2MGTI-MGL3BXN-PQ6W5BM-TBBZ4TJ-XZWICQ2"></device>
<minDiskFreePct>1</minDiskFreePct>
<maxConflicts>-1</maxConflicts>
</folder>
<device id="AIR6LPZ-7K4PTTV-UXQSMUU-CPQ5YWH-OEDFIIQ-JUG777G-2YQXXR5-YD6AWQR" name="node one" compression="metadata">
<address>tcp://a</address>
</device>
<device id="P56IOI7-MZJNU2Y-IQGDREY-DM2MGTI-MGL3BXN-PQ6W5BM-TBBZ4TJ-XZWICQ2" name="node two" compression="metadata">
<address>tcp://b</address>
</device>
</configuration>
+15 -5
View File
@@ -21,7 +21,7 @@ import (
const relayPriority = 200
func init() {
dialers["relay"] = newRelayDialer
dialers["relay"] = relayDialerFactory{}
}
type relayDialer struct {
@@ -70,13 +70,23 @@ func (d *relayDialer) RedialFrequency() time.Duration {
return time.Duration(d.cfg.Options().RelayReconnectIntervalM) * time.Minute
}
func (d *relayDialer) String() string {
return "Relay Dialer"
}
type relayDialerFactory struct{}
func newRelayDialer(cfg *config.Wrapper, tlsCfg *tls.Config) genericDialer {
func (relayDialerFactory) New(cfg *config.Wrapper, tlsCfg *tls.Config) genericDialer {
return &relayDialer{
cfg: cfg,
tlsCfg: tlsCfg,
}
}
func (relayDialerFactory) Priority() int {
return relayPriority
}
func (relayDialerFactory) Enabled(cfg config.Configuration) bool {
return cfg.Options.RelaysEnabled
}
func (relayDialerFactory) String() string {
return "Relay Dialer"
}
+24 -10
View File
@@ -13,23 +13,26 @@ import (
"sync"
"time"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/dialer"
"github.com/syncthing/syncthing/lib/nat"
"github.com/syncthing/syncthing/lib/relay/client"
)
func init() {
listeners["relay"] = newRelayListener
listeners["dynamic+http"] = newRelayListener
listeners["dynamic+https"] = newRelayListener
factory := &relayListenerFactory{}
listeners["relay"] = factory
listeners["dynamic+http"] = factory
listeners["dynamic+https"] = factory
}
type relayListener struct {
onAddressesChangedNotifier
uri *url.URL
tlsCfg *tls.Config
conns chan IntermediateConnection
uri *url.URL
tlsCfg *tls.Config
conns chan IntermediateConnection
factory listenerFactory
err error
client client.RelayClient
@@ -154,14 +157,25 @@ func (t *relayListener) Error() error {
return cerr
}
func (t *relayListener) Factory() listenerFactory {
return t.factory
}
func (t *relayListener) String() string {
return t.uri.String()
}
func newRelayListener(uri *url.URL, tlsCfg *tls.Config, conns chan IntermediateConnection, natService *nat.Service) genericListener {
type relayListenerFactory struct{}
func (f *relayListenerFactory) New(uri *url.URL, cfg *config.Wrapper, tlsCfg *tls.Config, conns chan IntermediateConnection, natService *nat.Service) genericListener {
return &relayListener{
uri: uri,
tlsCfg: tlsCfg,
conns: conns,
uri: uri,
tlsCfg: tlsCfg,
conns: conns,
factory: f,
}
}
func (relayListenerFactory) Enabled(cfg config.Configuration) bool {
return cfg.Options.RelaysEnabled
}
+145 -81
View File
@@ -9,6 +9,7 @@ package connections
import (
"crypto/tls"
"encoding/binary"
"errors"
"fmt"
"io"
"net"
@@ -54,9 +55,11 @@ type Service struct {
natService *nat.Service
natServiceToken *suture.ServiceToken
mut sync.RWMutex
listeners map[string]genericListener
listenerTokens map[string]suture.ServiceToken
listenersMut sync.RWMutex
listeners map[string]genericListener
listenerTokens map[string]suture.ServiceToken
curConMut sync.Mutex
currentConnection map[protocol.DeviceID]Connection
}
@@ -76,9 +79,11 @@ func NewService(cfg *config.Wrapper, myID protocol.DeviceID, mdl Model, tlsCfg *
lans: lans,
natService: nat.NewService(myID, cfg),
mut: sync.NewRWMutex(),
listeners: make(map[string]genericListener),
listenerTokens: make(map[string]suture.ServiceToken),
listenersMut: sync.NewRWMutex(),
listeners: make(map[string]genericListener),
listenerTokens: make(map[string]suture.ServiceToken),
curConMut: sync.NewMutex(),
currentConnection: make(map[protocol.DeviceID]Connection),
}
cfg.Subscribe(service)
@@ -108,6 +113,10 @@ func NewService(cfg *config.Wrapper, myID protocol.DeviceID, mdl Model, tlsCfg *
return service
}
var (
errDisabled = errors.New("disabled by configuration")
)
func (s *Service) handle() {
next:
for c := range s.conns {
@@ -153,9 +162,9 @@ next:
// If we have a relay connection, and the new incoming connection is
// not a relay connection, we should drop that, and prefer the this one.
s.mut.RLock()
skip := false
s.curConMut.Lock()
ct, ok := s.currentConnection[remoteID]
s.curConMut.Unlock()
// Lower priority is better, just like nice etc.
if ok && ct.Priority > c.Priority {
@@ -170,14 +179,10 @@ next:
// connections still established...
l.Infof("Connected to already connected device (%s)", remoteID)
c.Close()
skip = true
continue
} else if s.model.IsPaused(remoteID) {
l.Infof("Connection from paused device (%s)", remoteID)
c.Close()
skip = true
}
s.mut.RUnlock()
if skip {
continue
}
@@ -222,10 +227,10 @@ next:
l.Infof("Established secure connection to %s at %s", remoteID, name)
l.Debugf("cipher suite: %04X in lan: %t", c.ConnectionState().CipherSuite, !limit)
s.mut.Lock()
s.model.AddConnection(modelConn, hello)
s.curConMut.Lock()
s.currentConnection[remoteID] = modelConn
s.mut.Unlock()
s.curConMut.Unlock()
continue next
}
}
@@ -237,32 +242,56 @@ next:
func (s *Service) connect() {
nextDial := make(map[string]time.Time)
delay := time.Second
sleep := time.Second
// Used as delay for the first few connection attempts, increases
// exponentially
initialRampup := time.Second
// Calculated from actual dialers reconnectInterval
var sleep time.Duration
for {
cfg := s.cfg.Raw()
bestDialerPrio := 1<<31 - 1 // worse prio won't build on 32 bit
for _, df := range dialers {
if !df.Enabled(cfg) {
continue
}
if prio := df.Priority(); prio < bestDialerPrio {
bestDialerPrio = prio
}
}
l.Debugln("Reconnect loop")
now := time.Now()
var seen []string
nextDevice:
for deviceID, deviceCfg := range s.cfg.Devices() {
for _, deviceCfg := range cfg.Devices {
deviceID := deviceCfg.DeviceID
if deviceID == s.myID {
continue
}
l.Debugln("Reconnect loop for", deviceID)
s.mut.RLock()
paused := s.model.IsPaused(deviceID)
connected := s.model.ConnectedTo(deviceID)
ct := s.currentConnection[deviceID]
s.mut.RUnlock()
if paused {
continue
}
connected := s.model.ConnectedTo(deviceID)
s.curConMut.Lock()
ct := s.currentConnection[deviceID]
s.curConMut.Unlock()
if connected && ct.Priority == bestDialerPrio {
// Things are already as good as they can get.
continue
}
l.Debugln("Reconnect loop for", deviceID)
var addrs []string
for _, addr := range deviceCfg.Addresses {
if addr == "dynamic" {
@@ -279,35 +308,40 @@ func (s *Service) connect() {
seen = append(seen, addrs...)
for _, addr := range addrs {
nextDialAt, ok := nextDial[addr]
if ok && initialRampup >= sleep && nextDialAt.After(now) {
l.Debugf("Not dialing %v as sleep is %v, next dial is at %s and current time is %s", addr, sleep, nextDialAt, now)
continue
}
// If we fail at any step before actually getting the dialer
// retry in a minute
nextDial[addr] = now.Add(time.Minute)
uri, err := url.Parse(addr)
if err != nil {
l.Infoln("Failed to parse connection url:", addr, err)
l.Infof("Dialer for %s: %v", addr, err)
continue
}
dialerFactory, ok := dialers[uri.Scheme]
if !ok {
l.Debugln("Unknown address schema", uri)
dialerFactory, err := s.getDialerFactory(cfg, uri)
if err == errDisabled {
l.Debugln("Dialer for", uri, "is disabled")
continue
}
if err != nil {
l.Infof("Dialer for %v: %v", uri, err)
continue
}
dialer := dialerFactory(s.cfg, s.tlsCfg)
nextDialAt, ok := nextDial[uri.String()]
// See below for comments on this delay >= sleep check
if delay >= sleep && ok && nextDialAt.After(now) {
l.Debugf("Not dialing as next dial is at %s and current time is %s", nextDialAt, now)
continue
}
nextDial[uri.String()] = now.Add(dialer.RedialFrequency())
if connected && dialer.Priority() >= ct.Priority {
l.Debugf("Not dialing using %s as priorty is less than current connection (%d >= %d)", dialer, dialer.Priority(), ct.Priority)
if connected && dialerFactory.Priority() >= ct.Priority {
l.Debugf("Not dialing using %s as priorty is less than current connection (%d >= %d)", dialerFactory, dialerFactory.Priority(), ct.Priority)
continue
}
dialer := dialerFactory.New(s.cfg, s.tlsCfg)
l.Debugln("dial", deviceCfg.DeviceID, uri)
nextDial[addr] = now.Add(dialer.RedialFrequency())
conn, err := dialer.Dial(deviceID, uri)
if err != nil {
l.Debugln("dial failed", deviceCfg.DeviceID, uri, err)
@@ -325,12 +359,12 @@ func (s *Service) connect() {
nextDial, sleep = filterAndFindSleepDuration(nextDial, seen, now)
// delay variable is used to trigger much more frequent dialing after
// initial startup, essentially causing redials every 1, 2, 4, 8... seconds
if delay < sleep {
time.Sleep(delay)
delay *= 2
if initialRampup < sleep {
l.Debugln("initial rampup; sleep", initialRampup, "and update to", initialRampup*2)
time.Sleep(initialRampup)
initialRampup *= 2
} else {
l.Debugln("sleep until next dial", sleep)
time.Sleep(sleep)
}
}
@@ -353,25 +387,16 @@ func (s *Service) shouldLimit(addr net.Addr) bool {
return !tcpaddr.IP.IsLoopback()
}
func (s *Service) createListener(addr string) {
uri, err := url.Parse(addr)
if err != nil {
l.Infoln("Failed to parse listen address:", addr, err)
return
}
func (s *Service) createListener(factory listenerFactory, uri *url.URL) bool {
// must be called with listenerMut held
listenerFactory, ok := listeners[uri.Scheme]
if !ok {
l.Infoln("Unknown listen address scheme:", uri.String())
return
}
l.Debugln("Starting listener", uri)
listener := listenerFactory(uri, s.tlsCfg, s.conns, s.natService)
listener := factory.New(uri, s.cfg, s.tlsCfg, s.conns, s.natService)
listener.OnAddressesChanged(s.logListenAddressesChangedEvent)
s.mut.Lock()
s.listeners[addr] = listener
s.listenerTokens[addr] = s.Add(listener)
s.mut.Unlock()
s.listeners[uri.String()] = listener
s.listenerTokens[uri.String()] = s.Add(listener)
return true
}
func (s *Service) logListenAddressesChangedEvent(l genericListener) {
@@ -402,30 +427,43 @@ func (s *Service) CommitConfiguration(from, to config.Configuration) bool {
}
}
s.mut.RLock()
existingListeners := s.listeners
s.mut.RUnlock()
s.listenersMut.Lock()
seen := make(map[string]struct{})
for _, addr := range config.Wrap("", to).ListenAddresses() {
if _, ok := existingListeners[addr]; !ok {
l.Debugln("Staring listener", addr)
s.createListener(addr)
if _, ok := s.listeners[addr]; ok {
seen[addr] = struct{}{}
continue
}
uri, err := url.Parse(addr)
if err != nil {
l.Infof("Listener for %s: %v", addr, err)
continue
}
factory, err := s.getListenerFactory(to, uri)
if err == errDisabled {
l.Debugln("Listener for", uri, "is disabled")
continue
}
if err != nil {
l.Infof("Listener for %v: %v", uri, err)
continue
}
s.createListener(factory, uri)
seen[addr] = struct{}{}
}
s.mut.Lock()
for addr := range s.listeners {
if _, ok := seen[addr]; !ok {
for addr, listener := range s.listeners {
if _, ok := seen[addr]; !ok || !listener.Factory().Enabled(to) {
l.Debugln("Stopping listener", addr)
s.Remove(s.listenerTokens[addr])
delete(s.listenerTokens, addr)
delete(s.listeners, addr)
}
}
s.mut.Unlock()
s.listenersMut.Unlock()
if to.Options.NATEnabled && s.natServiceToken == nil {
l.Debugln("Starting NAT service")
@@ -441,7 +479,7 @@ func (s *Service) CommitConfiguration(from, to config.Configuration) bool {
}
func (s *Service) AllAddresses() []string {
s.mut.RLock()
s.listenersMut.RLock()
var addrs []string
for _, listener := range s.listeners {
for _, lanAddr := range listener.LANAddresses() {
@@ -451,24 +489,24 @@ func (s *Service) AllAddresses() []string {
addrs = append(addrs, wanAddr.String())
}
}
s.mut.RUnlock()
s.listenersMut.RUnlock()
return util.UniqueStrings(addrs)
}
func (s *Service) ExternalAddresses() []string {
s.mut.RLock()
s.listenersMut.RLock()
var addrs []string
for _, listener := range s.listeners {
for _, wanAddr := range listener.WANAddresses() {
addrs = append(addrs, wanAddr.String())
}
}
s.mut.RUnlock()
s.listenersMut.RUnlock()
return util.UniqueStrings(addrs)
}
func (s *Service) Status() map[string]interface{} {
s.mut.RLock()
s.listenersMut.RLock()
result := make(map[string]interface{})
for addr, listener := range s.listeners {
status := make(map[string]interface{})
@@ -483,10 +521,36 @@ func (s *Service) Status() map[string]interface{} {
result[addr] = status
}
s.mut.RUnlock()
s.listenersMut.RUnlock()
return result
}
func (s *Service) getDialerFactory(cfg config.Configuration, uri *url.URL) (dialerFactory, error) {
dialerFactory, ok := dialers[uri.Scheme]
if !ok {
return nil, fmt.Errorf("unknown address scheme %q", uri.Scheme)
}
if !dialerFactory.Enabled(cfg) {
return nil, errDisabled
}
return dialerFactory, nil
}
func (s *Service) getListenerFactory(cfg config.Configuration, uri *url.URL) (listenerFactory, error) {
listenerFactory, ok := listeners[uri.Scheme]
if !ok {
return nil, fmt.Errorf("unknown address scheme %q", uri.Scheme)
}
if !listenerFactory.Enabled(cfg) {
return nil, errDisabled
}
return listenerFactory, nil
}
func exchangeHello(c net.Conn, h protocol.HelloMessage) (protocol.HelloMessage, error) {
if err := c.SetDeadline(time.Now().Add(2 * time.Second)); err != nil {
return protocol.HelloMessage{}, err
+13 -6
View File
@@ -28,16 +28,22 @@ type Connection struct {
protocol.Connection
}
type dialerFactory func(*config.Wrapper, *tls.Config) genericDialer
type genericDialer interface {
Dial(protocol.DeviceID, *url.URL) (IntermediateConnection, error)
type dialerFactory interface {
New(*config.Wrapper, *tls.Config) genericDialer
Priority() int
RedialFrequency() time.Duration
Enabled(config.Configuration) bool
String() string
}
type listenerFactory func(*url.URL, *tls.Config, chan IntermediateConnection, *nat.Service) genericListener
type genericDialer interface {
Dial(protocol.DeviceID, *url.URL) (IntermediateConnection, error)
RedialFrequency() time.Duration
}
type listenerFactory interface {
New(*url.URL, *config.Wrapper, *tls.Config, chan IntermediateConnection, *nat.Service) genericListener
Enabled(config.Configuration) bool
}
type genericListener interface {
Serve()
@@ -55,6 +61,7 @@ type genericListener interface {
Error() error
OnAddressesChanged(func(genericListener))
String() string
Factory() listenerFactory
}
type Model interface {
+16 -9
View File
@@ -20,8 +20,9 @@ import (
const tcpPriority = 10
func init() {
factory := &tcpDialerFactory{}
for _, scheme := range []string{"tcp", "tcp4", "tcp6"} {
dialers[scheme] = newTCPDialer
dialers[scheme] = factory
}
}
@@ -55,21 +56,27 @@ func (d *tcpDialer) Dial(id protocol.DeviceID, uri *url.URL) (IntermediateConnec
return IntermediateConnection{tc, "TCP (Client)", tcpPriority}, nil
}
func (tcpDialer) Priority() int {
return tcpPriority
}
func (d *tcpDialer) RedialFrequency() time.Duration {
return time.Duration(d.cfg.Options().ReconnectIntervalS) * time.Second
}
func (d *tcpDialer) String() string {
return "TCP Dialer"
}
type tcpDialerFactory struct{}
func newTCPDialer(cfg *config.Wrapper, tlsCfg *tls.Config) genericDialer {
func (tcpDialerFactory) New(cfg *config.Wrapper, tlsCfg *tls.Config) genericDialer {
return &tcpDialer{
cfg: cfg,
tlsCfg: tlsCfg,
}
}
func (tcpDialerFactory) Priority() int {
return tcpPriority
}
func (tcpDialerFactory) Enabled(cfg config.Configuration) bool {
return true
}
func (tcpDialerFactory) String() string {
return "TCP Dialer"
}
+25 -9
View File
@@ -14,30 +14,32 @@ import (
"sync"
"time"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/dialer"
"github.com/syncthing/syncthing/lib/nat"
)
func init() {
factory := &tcpListenerFactory{}
for _, scheme := range []string{"tcp", "tcp4", "tcp6"} {
listeners[scheme] = newTCPListener
listeners[scheme] = factory
}
}
type tcpListener struct {
onAddressesChangedNotifier
uri *url.URL
tlsCfg *tls.Config
stop chan struct{}
conns chan IntermediateConnection
uri *url.URL
tlsCfg *tls.Config
stop chan struct{}
conns chan IntermediateConnection
factory listenerFactory
natService *nat.Service
mapping *nat.Mapping
address *url.URL
err error
mut sync.RWMutex
err error
mut sync.RWMutex
}
func (t *tcpListener) Serve() {
@@ -64,6 +66,9 @@ func (t *tcpListener) Serve() {
}
defer listener.Close()
l.Infof("TCP listener (%v) starting", listener.Addr())
defer l.Infof("TCP listener (%v) shutting down", listener.Addr())
mapping := t.natService.NewMapping(nat.TCP, tcaddr.IP, tcaddr.Port)
mapping.OnChanged(func(_ *nat.Mapping, _, _ []nat.Address) {
t.notifyAddressesChanged(t)
@@ -153,16 +158,27 @@ func (t *tcpListener) String() string {
return t.uri.String()
}
func newTCPListener(uri *url.URL, tlsCfg *tls.Config, conns chan IntermediateConnection, natService *nat.Service) genericListener {
func (t *tcpListener) Factory() listenerFactory {
return t.factory
}
type tcpListenerFactory struct{}
func (f *tcpListenerFactory) New(uri *url.URL, cfg *config.Wrapper, tlsCfg *tls.Config, conns chan IntermediateConnection, natService *nat.Service) genericListener {
return &tcpListener{
uri: fixupPort(uri),
tlsCfg: tlsCfg,
conns: conns,
natService: natService,
stop: make(chan struct{}),
factory: f,
}
}
func (tcpListenerFactory) Enabled(cfg config.Configuration) bool {
return true
}
func isPublicIPv4(ip net.IP) bool {
ip = ip.To4()
if ip == nil {
+20
View File
@@ -262,7 +262,17 @@ func (db *Instance) withHave(folder, device, prefix []byte, truncate bool, fn It
dbi := t.NewIterator(util.BytesPrefix(db.deviceKey(folder, device, prefix)[:keyPrefixLen+keyFolderLen+keyDeviceLen+len(prefix)]), nil)
defer dbi.Release()
slashedPrefix := prefix
if !bytes.HasSuffix(prefix, []byte{'/'}) {
slashedPrefix = append(slashedPrefix, '/')
}
for dbi.Next() {
name := db.deviceKeyName(dbi.Key())
if len(prefix) > 0 && !bytes.Equal(name, prefix) && !bytes.HasPrefix(name, slashedPrefix) {
return
}
// The iterator function may keep a reference to the unmarshalled
// struct, which in turn references the buffer it was unmarshalled
// from. dbi.Value() just returns an internal slice that it reuses, so
@@ -359,6 +369,11 @@ func (db *Instance) withGlobal(folder, prefix []byte, truncate bool, fn Iterator
dbi := t.NewIterator(util.BytesPrefix(db.globalKey(folder, prefix)), nil)
defer dbi.Release()
slashedPrefix := prefix
if !bytes.HasSuffix(prefix, []byte{'/'}) {
slashedPrefix = append(slashedPrefix, '/')
}
var fk []byte
for dbi.Next() {
var vl versionList
@@ -370,7 +385,12 @@ func (db *Instance) withGlobal(folder, prefix []byte, truncate bool, fn Iterator
l.Debugln(dbi.Key())
panic("no versions?")
}
name := db.globalKeyName(dbi.Key())
if len(prefix) > 0 && !bytes.Equal(name, prefix) && !bytes.HasPrefix(name, slashedPrefix) {
return
}
fk = db.deviceKeyInto(fk[:cap(fk)], folder, vl.versions[0].device, name)
bs, err := t.Get(fk, nil)
if err != nil {
+1 -1
View File
@@ -18,7 +18,7 @@ import (
// The CachingMux aggregates results from multiple Finders. Each Finder has
// an associated cache time and negative cache time. The cache time sets how
// long we cache and return successfull lookup results, the negative cache
// long we cache and return successful lookup results, the negative cache
// time sets how long we refrain from asking about the same device ID after
// receiving a negative answer. The value of zero disables caching (positive
// or negative).
+8 -2
View File
@@ -12,6 +12,7 @@ import (
"encoding/json"
"errors"
"io"
"io/ioutil"
"net/http"
"net/url"
"strconv"
@@ -154,9 +155,14 @@ func (c *globalClient) Lookup(device protocol.DeviceID) (addresses []string, err
return nil, err
}
var ann announcement
err = json.NewDecoder(resp.Body).Decode(&ann)
bs, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
resp.Body.Close()
var ann announcement
err = json.Unmarshal(bs, &ann)
return ann.Addresses, err
}
+5 -5
View File
@@ -280,14 +280,14 @@ func parseIgnoreFile(fd io.Reader, currentFile string, seen map[string]bool) ([]
// Pattern is rooted in the current dir only
pattern.match, err = glob.Compile(line[1:])
if err != nil {
return fmt.Errorf("invalid pattern %q in ignore file", line)
return fmt.Errorf("invalid pattern %q in ignore file (%v)", line, err)
}
patterns = append(patterns, pattern)
} else if strings.HasPrefix(line, "**/") {
// Add the pattern as is, and without **/ so it matches in current dir
pattern.match, err = glob.Compile(line)
if err != nil {
return fmt.Errorf("invalid pattern %q in ignore file", line)
return fmt.Errorf("invalid pattern %q in ignore file (%v)", line, err)
}
patterns = append(patterns, pattern)
@@ -295,7 +295,7 @@ func parseIgnoreFile(fd io.Reader, currentFile string, seen map[string]bool) ([]
pattern.pattern = line
pattern.match, err = glob.Compile(line)
if err != nil {
return fmt.Errorf("invalid pattern %q in ignore file", line)
return fmt.Errorf("invalid pattern %q in ignore file (%v)", line, err)
}
patterns = append(patterns, pattern)
} else if strings.HasPrefix(line, "#include ") {
@@ -311,7 +311,7 @@ func parseIgnoreFile(fd io.Reader, currentFile string, seen map[string]bool) ([]
// current directory and subdirs.
pattern.match, err = glob.Compile(line)
if err != nil {
return fmt.Errorf("invalid pattern %q in ignore file", line)
return fmt.Errorf("invalid pattern %q in ignore file (%v)", line, err)
}
patterns = append(patterns, pattern)
@@ -319,7 +319,7 @@ func parseIgnoreFile(fd io.Reader, currentFile string, seen map[string]bool) ([]
pattern.pattern = line
pattern.match, err = glob.Compile(line)
if err != nil {
return fmt.Errorf("invalid pattern %q in ignore file", line)
return fmt.Errorf("invalid pattern %q in ignore file (%v)", line, err)
}
patterns = append(patterns, pattern)
}
+32 -2
View File
@@ -295,7 +295,7 @@ func TestCaching(t *testing.T) {
pats.Match(letter)
}
// Verify that outcomes preserved on next laod
// Verify that outcomes preserved on next load
err = pats.Load(fd1.Name())
if err != nil {
@@ -323,7 +323,7 @@ func TestCaching(t *testing.T) {
pats.Match(letter)
}
// Verify that outcomes provided on next laod
// Verify that outcomes provided on next load
err = pats.Load(fd1.Name())
if err != nil {
@@ -635,3 +635,33 @@ func TestAutomaticCaseInsensitivity(t *testing.T) {
}
}
}
func TestCommas(t *testing.T) {
stignore := `
foo,bar.txt
{baz,quux}.txt
`
pats := New(true)
err := pats.Parse(bytes.NewBufferString(stignore), ".stignore")
if err != nil {
t.Fatal(err)
}
tests := []struct {
name string
match bool
}{
{"foo.txt", false},
{"bar.txt", false},
{"foo,bar.txt", true},
{"baz.txt", true},
{"quux.txt", true},
{"baz,quux.txt", false},
}
for _, tc := range tests {
if pats.Match(tc.name).IsIgnored() != tc.match {
t.Errorf("Match of %s was %v, should be %v", tc.name, !tc.match, tc.match)
}
}
}
+2 -2
View File
@@ -28,7 +28,7 @@ type deviceFolderDownloadState struct {
numberOfBlocksInProgress int
}
// Has returns wether a block at that specific index, and that specific version of the file
// Has returns whether a block at that specific index, and that specific version of the file
// is currently available on the remote device for pulling from a temporary file.
func (p *deviceFolderDownloadState) Has(file string, version protocol.Vector, index int32) bool {
p.mut.RLock()
@@ -117,7 +117,7 @@ func (t *deviceDownloadState) Update(folder string, updates []protocol.FileDownl
f.Update(updates)
}
// Has returns wether block at that specific index, and that specific version of the file
// Has returns whether block at that specific index, and that specific version of the file
// is currently available on the remote device for pulling from a temporary file.
func (t *deviceDownloadState) Has(folder, file string, version protocol.Vector, index int32) bool {
if t == nil {
+7 -6
View File
@@ -1048,7 +1048,7 @@ func (m *Model) AddConnection(conn connections.Connection, hello protocol.HelloM
m.pmut.Unlock()
device, ok := m.cfg.Devices()[deviceID]
if ok && (device.Name == "" || m.cfg.Options().OverwriteNames) {
if ok && (device.Name == "" || m.cfg.Options().OverwriteRemoteDevNames) {
device.Name = hello.DeviceName
m.cfg.SetDevice(device)
m.cfg.Save()
@@ -1401,7 +1401,9 @@ func (m *Model) internalScanFolderSubdirs(folder string, subs []string) error {
cancel := make(chan struct{})
defer close(cancel)
w := &scanner.Walker{
runner.setState(FolderScanning)
fchan, err := scanner.Walk(scanner.Config{
Folder: folderCfg.ID,
Dir: folderCfg.Path(),
Subs: subs,
@@ -1417,11 +1419,8 @@ func (m *Model) internalScanFolderSubdirs(folder string, subs []string) error {
ShortID: m.shortID,
ProgressTickIntervalS: folderCfg.ScanProgressIntervalS,
Cancel: cancel,
}
})
runner.setState(FolderScanning)
fchan, err := w.Walk()
if err != nil {
// The error we get here is likely an OS level error, which might not be
// as readable as our health check errors. Check if we can get a health
@@ -2018,6 +2017,8 @@ func (m *Model) CommitConfiguration(from, to config.Configuration) bool {
// by themselves.
from.Options.URAccepted = to.Options.URAccepted
from.Options.URUniqueID = to.Options.URUniqueID
from.Options.ListenAddresses = to.Options.ListenAddresses
from.Options.RelaysEnabled = to.Options.RelaysEnabled
// All of the other generic options require restart. Or at least they may;
// removing this check requires going through those options carefully and
// making sure there are individual services that handle them correctly.
+62 -2
View File
@@ -386,7 +386,7 @@ func TestDeviceRename(t *testing.T) {
m.Close(device1, protocol.ErrTimeout)
opts := cfg.Options()
opts.OverwriteNames = true
opts.OverwriteRemoteDevNames = true
cfg.SetOptions(opts)
hello.DeviceName = "tester2"
@@ -1308,7 +1308,7 @@ func TestUnifySubs(t *testing.T) {
[]string{".stfolder", ".stignore"},
},
{
// 7. but the presense of something else unknown forces an actual
// 7. but the presence of something else unknown forces an actual
// scan
[]string{".stfolder", ".stignore", "foo/bar"},
[]string{},
@@ -1360,6 +1360,66 @@ func TestUnifySubs(t *testing.T) {
}
}
func TestIssue3028(t *testing.T) {
// Create two files that we'll delete, one with a name that is a prefix of the other.
if err := ioutil.WriteFile("testdata/testrm", []byte("Hello"), 0644); err != nil {
t.Fatal(err)
}
defer os.Remove("testdata/testrm")
if err := ioutil.WriteFile("testdata/testrm2", []byte("Hello"), 0644); err != nil {
t.Fatal(err)
}
defer os.Remove("testdata/testrm2")
// Create a model and default folder
db := db.OpenMemory()
m := NewModel(defaultConfig, protocol.LocalDeviceID, "device", "syncthing", "dev", db, nil)
defCfg := defaultFolderConfig.Copy()
defCfg.RescanIntervalS = 86400
m.AddFolder(defCfg)
m.StartFolder("default")
m.ServeBackground()
// Ugly hack for testing: reach into the model for the rwfolder and wait
// for it to complete the initial scan. The risk is that it otherwise
// runs during our modifications and screws up the test.
m.fmut.RLock()
folder := m.folderRunners["default"].(*rwFolder)
m.fmut.RUnlock()
<-folder.initialScanCompleted
// Get a count of how many files are there now
locorigfiles, _, _ := m.LocalSize("default")
globorigfiles, _, _ := m.GlobalSize("default")
// Delete and rescan specifically these two
os.Remove("testdata/testrm")
os.Remove("testdata/testrm2")
m.ScanFolderSubs("default", []string{"testrm", "testrm2"})
// Verify that the number of files decreased by two and the number of
// deleted files increases by two
locnowfiles, locdelfiles, _ := m.LocalSize("default")
globnowfiles, globdelfiles, _ := m.GlobalSize("default")
if locnowfiles != locorigfiles-2 {
t.Errorf("Incorrect local accounting; got %d current files, expected %d", locnowfiles, locorigfiles-2)
}
if globnowfiles != globorigfiles-2 {
t.Errorf("Incorrect global accounting; got %d current files, expected %d", globnowfiles, globorigfiles-2)
}
if locdelfiles != 2 {
t.Errorf("Incorrect local accounting; got %d deleted files, expected 2", locdelfiles)
}
if globdelfiles != 2 {
t.Errorf("Incorrect global accounting; got %d deleted files, expected 2", globdelfiles)
}
}
type fakeAddr struct{}
func (fakeAddr) Network() string {
+12 -6
View File
@@ -100,6 +100,8 @@ type rwFolder struct {
errors map[string]string // path -> error string
errorsMut sync.Mutex
initialScanCompleted chan (struct{}) // exposed for testing
}
func newRWFolder(model *Model, cfg config.FolderConfiguration, ver versioner.Versioner) service {
@@ -135,6 +137,8 @@ func newRWFolder(model *Model, cfg config.FolderConfiguration, ver versioner.Ver
remoteIndex: make(chan struct{}, 1), // This needs to be 1-buffered so that we queue a notification if we're busy doing a pull when it comes.
errorsMut: sync.NewMutex(),
initialScanCompleted: make(chan struct{}),
}
f.configureCopiersAndPullers(cfg)
@@ -186,9 +190,6 @@ func (f *rwFolder) Serve() {
var prevVer int64
var prevIgnoreHash string
// We don't start pulling files until a scan has been completed.
initialScanCompleted := false
for {
select {
case <-f.stop:
@@ -200,7 +201,10 @@ func (f *rwFolder) Serve() {
l.Debugln(f, "remote index updated, rescheduling pull")
case <-f.pullTimer.C:
if !initialScanCompleted {
select {
case <-f.initialScanCompleted:
default:
// We don't start pulling files until a scan has been completed.
l.Debugln(f, "skip (initial)")
f.pullTimer.Reset(f.sleep)
continue
@@ -297,9 +301,11 @@ func (f *rwFolder) Serve() {
if err != nil {
continue
}
if !initialScanCompleted {
select {
case <-f.initialScanCompleted:
default:
l.Infoln("Completed initial scan (rw) of folder", f.folderID)
initialScanCompleted = true
close(f.initialScanCompleted)
}
case req := <-f.scan.now:
+1 -1
View File
@@ -20,7 +20,7 @@ var (
)
// An AtomicWriter is an *os.File that writes to a temporary file in the same
// directory as the final path. On successfull Close the file is renamed to
// directory as the final path. On successful Close the file is renamed to
// it's final path. Any error on Write or during Close is accumulated and
// returned on Close, so a lazy user can ignore errors until Close.
type AtomicWriter struct {
+1 -1
View File
@@ -14,7 +14,7 @@ import (
)
// TCPPing returns the duration required to establish a TCP connection
// to the given host. ICMP packets require root priviledges, hence why we use
// to the given host. ICMP packets require root privileges, hence why we use
// tcp.
func TCPPing(address string) (time.Duration, error) {
start := time.Now()
+2 -2
View File
@@ -10,7 +10,7 @@ package osutil
import "syscall"
// MaximizeOpenFileLimit tries to set the resoure limit RLIMIT_NOFILE (number
// MaximizeOpenFileLimit tries to set the resource limit RLIMIT_NOFILE (number
// of open file descriptors) to the max (hard limit), if the current (soft
// limit) is below the max. Returns the new (though possibly unchanged) limit,
// or an error if it could not be changed.
@@ -35,7 +35,7 @@ func MaximizeOpenFileLimit() (int, error) {
// If the set succeeded, perform a new get to see what happened. We might
// have gotten a value lower than the one in lim.Max, if lim.Max was
// something that indiciated "unlimited" (i.e. intmax).
// something that indicated "unlimited" (i.e. intmax).
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim); err != nil {
// We don't really know the correct value here since Getrlimit
// mysteriously failed after working once... Shouldn't ever happen, I
+188
View File
@@ -0,0 +1,188 @@
// Copyright (C) 2016 The Protocol Authors.
package protocol
import (
"crypto/tls"
"encoding/binary"
"net"
"testing"
"github.com/syncthing/syncthing/lib/dialer"
)
func BenchmarkRequestsRawTCP(b *testing.B) {
// Benchmarks the rate at which we can serve requests over a single,
// unencrypted TCP channel over the loopback interface.
// Get a connected TCP pair
conn0, conn1, err := getTCPConnectionPair()
if err != nil {
b.Fatal(err)
}
defer conn0.Close()
defer conn1.Close()
// Bench it
benchmarkRequestsConnPair(b, conn0, conn1)
}
func BenchmarkRequestsTLSoTCP(b *testing.B) {
// Benchmarks the rate at which we can serve requests over a single,
// TLS encrypted TCP channel over the loopback interface.
// Load a certificate, skipping this benchmark if it doesn't exist
cert, err := tls.LoadX509KeyPair("../../test/h1/cert.pem", "../../test/h1/key.pem")
if err != nil {
b.Skip(err)
return
}
// Get a connected TCP pair
conn0, conn1, err := getTCPConnectionPair()
if err != nil {
b.Fatal(err)
}
/// TLSify them
conn0, conn1 = negotiateTLS(cert, conn0, conn1)
defer conn0.Close()
defer conn1.Close()
// Bench it
benchmarkRequestsConnPair(b, conn0, conn1)
}
func benchmarkRequestsConnPair(b *testing.B, conn0, conn1 net.Conn) {
// Start up Connections on them
c0 := NewConnection(LocalDeviceID, conn0, conn0, new(fakeModel), "c0", CompressMetadata)
c0.Start()
c1 := NewConnection(LocalDeviceID, conn1, conn1, new(fakeModel), "c1", CompressMetadata)
c1.Start()
// Satisfy the assertions in the protocol by sending an initial cluster config
c0.ClusterConfig(ClusterConfigMessage{})
c1.ClusterConfig(ClusterConfigMessage{})
// Report some useful stats and reset the timer for the actual test
b.ReportAllocs()
b.SetBytes(128 << 10)
b.ResetTimer()
// Request 128 KiB blocks, which will be satisfied by zero copy from the
// other side (we'll get back a full block of zeroes).
var buf []byte
var err error
for i := 0; i < b.N; i++ {
// Use c0 and c1 for each alternating request, so we get as much
// data flowing in both directions.
if i%2 == 0 {
buf, err = c0.Request("folder", "file", int64(i), 128<<10, nil, false)
} else {
buf, err = c1.Request("folder", "file", int64(i), 128<<10, nil, false)
}
if err != nil {
b.Fatal(err)
}
if len(buf) != 128<<10 {
b.Fatal("Incorrect returned buf length", len(buf), "!=", 128<<10)
}
// The fake model is supposed to tag the end of the buffer with the
// requested offset, so we can verify that we get back data for this
// block correctly.
if binary.BigEndian.Uint64(buf[128<<10-8:]) != uint64(i) {
b.Fatal("Bad data returned")
}
}
}
// returns the two endpoints of a TCP connection over lo0
func getTCPConnectionPair() (net.Conn, net.Conn, error) {
lst, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
return nil, nil, err
}
// We run the Accept in the background since it's blocking, and we use
// the channel to make the race thingies happy about writing vs reading
// conn0 and err0.
var conn0 net.Conn
var err0 error
done := make(chan struct{})
go func() {
conn0, err0 = lst.Accept()
close(done)
}()
// Dial the connection
conn1, err := net.Dial("tcp", lst.Addr().String())
if err != nil {
return nil, nil, err
}
// Check any error from accept
<-done
if err0 != nil {
return nil, nil, err0
}
// Set the buffer sizes etc as usual
dialer.SetTCPOptions(conn0.(*net.TCPConn))
dialer.SetTCPOptions(conn1.(*net.TCPConn))
return conn0, conn1, nil
}
func negotiateTLS(cert tls.Certificate, conn0, conn1 net.Conn) (net.Conn, net.Conn) {
cfg := &tls.Config{
Certificates: []tls.Certificate{cert},
NextProtos: []string{"bep/1.0"},
ClientAuth: tls.RequestClientCert,
SessionTicketsDisabled: true,
InsecureSkipVerify: true,
MinVersion: tls.VersionTLS12,
CipherSuites: []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
},
}
tlsc0 := tls.Server(conn0, cfg)
tlsc1 := tls.Client(conn1, cfg)
return tlsc0, tlsc1
}
// The fake model does nothing much
type fakeModel struct{}
func (m *fakeModel) Index(deviceID DeviceID, folder string, files []FileInfo, flags uint32, options []Option) {
}
func (m *fakeModel) IndexUpdate(deviceID DeviceID, folder string, files []FileInfo, flags uint32, options []Option) {
}
func (m *fakeModel) Request(deviceID DeviceID, folder string, name string, offset int64, hash []byte, flags uint32, options []Option, buf []byte) error {
// We write the offset to the end of the buffer, so the receiver
// can verify that it did in fact get some data back over the
// connection.
binary.BigEndian.PutUint64(buf[len(buf)-8:], uint64(offset))
return nil
}
func (m *fakeModel) ClusterConfig(deviceID DeviceID, config ClusterConfigMessage) {
}
func (m *fakeModel) Close(deviceID DeviceID, err error) {
}
func (m *fakeModel) DownloadProgress(deviceID DeviceID, folder string, updates []FileDownloadProgressUpdate, flags uint32, options []Option) {
}
+1 -1
View File
@@ -772,7 +772,7 @@ func (c *rawConnection) pingSender() {
}
}
// The pingReciever checks that we've received a message (any message will do,
// The pingReceiver checks that we've received a message (any message will do,
// but we expect pings in the absence of other messages) within the last
// ReceiveTimeout. If not, we close the connection with an ErrTimeout.
func (c *rawConnection) pingReceiver() {
Binary file not shown.
+1 -1
View File
@@ -182,7 +182,7 @@ func (c *dynamicClient) setError(err error) {
c.mut.Unlock()
}
// This is the announcement recieved from the relay server;
// This is the announcement received from the relay server;
// {"relays": [{"url": "relay://10.20.30.40:5060"}, ...]}
type dynamicAnnouncement struct {
Relays []struct {
+2 -1
View File
@@ -89,7 +89,8 @@ func (c *staticClient) Serve() {
return
}
l.Infoln("Joined relay", c.uri)
l.Infof("Joined relay %s://%s", c.uri.Scheme, c.uri.Host)
defer l.Infof("Disconnected from relay %s://%s", c.uri.Scheme, c.uri.Host)
c.mut.Lock()
c.connected = true
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"github.com/syncthing/syncthing/lib/sync"
)
// The parallell hasher reads FileInfo structures from the inbox, hashes the
// The parallel hasher reads FileInfo structures from the inbox, hashes the
// file to populate the Blocks element and sends it to the outbox. A number of
// workers are used in parallel. The outbox will become closed when the inbox
// is closed and all items handled.
+102 -70
View File
@@ -17,7 +17,6 @@ import (
"unicode/utf8"
"github.com/rcrowley/go-metrics"
"github.com/syncthing/syncthing/lib/db"
"github.com/syncthing/syncthing/lib/events"
"github.com/syncthing/syncthing/lib/ignore"
"github.com/syncthing/syncthing/lib/osutil"
@@ -41,7 +40,7 @@ func init() {
}
}
type Walker struct {
type Config struct {
// Folder for which the walker has been created
Folder string
// Dir is the base directory for the walk
@@ -58,8 +57,9 @@ type Walker struct {
TempLifetime time.Duration
// If CurrentFiler is not nil, it is queried for the current file before rescanning.
CurrentFiler CurrentFiler
// If MtimeRepo is not nil, it is used to provide mtimes on systems that don't support setting arbirtary mtimes.
MtimeRepo *db.VirtualMtimeRepo
// If MtimeRepo is not nil, it is used to provide mtimes on systems that
// don't support setting arbitrary mtimes.
MtimeRepo MtimeRepo
// If IgnorePerms is true, changes to permission bits will not be
// detected. Scanned files will get zero permission bits and the
// NoPermissionBits flag set.
@@ -79,8 +79,6 @@ type Walker struct {
}
type TempNamer interface {
// Temporary returns a temporary name for the filed referred to by filepath.
TempName(path string) string
// IsTemporary returns true if path refers to the name of temporary file.
IsTemporary(path string) bool
}
@@ -90,9 +88,35 @@ type CurrentFiler interface {
CurrentFile(name string) (protocol.FileInfo, bool)
}
type MtimeRepo interface {
// GetMtime returns a (possibly modified) actual mtime given a file name
// and its on disk mtime.
GetMtime(relPath string, mtime time.Time) time.Time
}
func Walk(cfg Config) (chan protocol.FileInfo, error) {
w := walker{cfg}
if w.CurrentFiler == nil {
w.CurrentFiler = noCurrentFiler{}
}
if w.TempNamer == nil {
w.TempNamer = noTempNamer{}
}
if w.MtimeRepo == nil {
w.MtimeRepo = noMtimeRepo{}
}
return w.walk()
}
type walker struct {
Config
}
// Walk returns the list of files found in the local folder by scanning the
// file system. Files are blockwise hashed.
func (w *Walker) Walk() (chan protocol.FileInfo, error) {
func (w *walker) walk() (chan protocol.FileInfo, error) {
l.Debugln("Walk", w.Dir, w.Subs, w.BlockSize, w.Matcher)
err := checkDir(w.Dir)
@@ -195,7 +219,7 @@ func (w *Walker) Walk() (chan protocol.FileInfo, error) {
return finishedChan, nil
}
func (w *Walker) walkAndHashFiles(fchan, dchan chan protocol.FileInfo) filepath.WalkFunc {
func (w *walker) walkAndHashFiles(fchan, dchan chan protocol.FileInfo) filepath.WalkFunc {
now := time.Now()
return func(absPath string, info os.FileInfo, err error) error {
// Return value used when we are returning early and don't want to
@@ -221,12 +245,9 @@ func (w *Walker) walkAndHashFiles(fchan, dchan chan protocol.FileInfo) filepath.
return nil
}
mtime := info.ModTime()
if w.MtimeRepo != nil {
mtime = w.MtimeRepo.GetMtime(relPath, mtime)
}
mtime := w.MtimeRepo.GetMtime(relPath, info.ModTime())
if w.TempNamer != nil && w.TempNamer.IsTemporary(relPath) {
if w.TempNamer.IsTemporary(relPath) {
// A temporary file
l.Debugln("temporary:", relPath)
if info.Mode().IsRegular() && mtime.Add(w.TempLifetime).Before(now) {
@@ -272,34 +293,30 @@ func (w *Walker) walkAndHashFiles(fchan, dchan chan protocol.FileInfo) filepath.
}
}
func (w *Walker) walkRegular(relPath string, info os.FileInfo, mtime time.Time, fchan chan protocol.FileInfo) error {
func (w *walker) walkRegular(relPath string, info os.FileInfo, mtime time.Time, fchan chan protocol.FileInfo) error {
curMode := uint32(info.Mode())
if runtime.GOOS == "windows" && osutil.IsWindowsExecutable(relPath) {
curMode |= 0111
}
var currentVersion protocol.Vector
if w.CurrentFiler != nil {
// A file is "unchanged", if it
// - exists
// - has the same permissions as previously, unless we are ignoring permissions
// - was not marked deleted (since it apparently exists now)
// - had the same modification time as it has now
// - was not a directory previously (since it's a file now)
// - was not a symlink (since it's a file now)
// - was not invalid (since it looks valid now)
// - has the same size as previously
cf, ok := w.CurrentFiler.CurrentFile(relPath)
permUnchanged := w.IgnorePerms || !cf.HasPermissionBits() || PermsEqual(cf.Flags, curMode)
if ok && permUnchanged && !cf.IsDeleted() && cf.Modified == mtime.Unix() && !cf.IsDirectory() &&
!cf.IsSymlink() && !cf.IsInvalid() && cf.Size() == info.Size() {
return nil
}
currentVersion = cf.Version
l.Debugln("rescan:", cf, mtime.Unix(), info.Mode()&os.ModePerm)
// A file is "unchanged", if it
// - exists
// - has the same permissions as previously, unless we are ignoring permissions
// - was not marked deleted (since it apparently exists now)
// - had the same modification time as it has now
// - was not a directory previously (since it's a file now)
// - was not a symlink (since it's a file now)
// - was not invalid (since it looks valid now)
// - has the same size as previously
cf, ok := w.CurrentFiler.CurrentFile(relPath)
permUnchanged := w.IgnorePerms || !cf.HasPermissionBits() || PermsEqual(cf.Flags, curMode)
if ok && permUnchanged && !cf.IsDeleted() && cf.Modified == mtime.Unix() && !cf.IsDirectory() &&
!cf.IsSymlink() && !cf.IsInvalid() && cf.Size() == info.Size() {
return nil
}
l.Debugln("rescan:", cf, mtime.Unix(), info.Mode()&os.ModePerm)
var flags = curMode & uint32(maskModePerm)
if w.IgnorePerms {
flags = protocol.FlagNoPermBits | 0666
@@ -307,7 +324,7 @@ func (w *Walker) walkRegular(relPath string, info os.FileInfo, mtime time.Time,
f := protocol.FileInfo{
Name: relPath,
Version: currentVersion.Update(w.ShortID),
Version: cf.Version.Update(w.ShortID),
Flags: flags,
Modified: mtime.Unix(),
CachedSize: info.Size(),
@@ -323,23 +340,18 @@ func (w *Walker) walkRegular(relPath string, info os.FileInfo, mtime time.Time,
return nil
}
func (w *Walker) walkDir(relPath string, info os.FileInfo, mtime time.Time, dchan chan protocol.FileInfo) error {
var currentVersion protocol.Vector
if w.CurrentFiler != nil {
// A directory is "unchanged", if it
// - exists
// - has the same permissions as previously, unless we are ignoring permissions
// - was not marked deleted (since it apparently exists now)
// - was a directory previously (not a file or something else)
// - was not a symlink (since it's a directory now)
// - was not invalid (since it looks valid now)
cf, ok := w.CurrentFiler.CurrentFile(relPath)
permUnchanged := w.IgnorePerms || !cf.HasPermissionBits() || PermsEqual(cf.Flags, uint32(info.Mode()))
if ok && permUnchanged && !cf.IsDeleted() && cf.IsDirectory() && !cf.IsSymlink() && !cf.IsInvalid() {
return nil
}
currentVersion = cf.Version
func (w *walker) walkDir(relPath string, info os.FileInfo, mtime time.Time, dchan chan protocol.FileInfo) error {
// A directory is "unchanged", if it
// - exists
// - has the same permissions as previously, unless we are ignoring permissions
// - was not marked deleted (since it apparently exists now)
// - was a directory previously (not a file or something else)
// - was not a symlink (since it's a directory now)
// - was not invalid (since it looks valid now)
cf, ok := w.CurrentFiler.CurrentFile(relPath)
permUnchanged := w.IgnorePerms || !cf.HasPermissionBits() || PermsEqual(cf.Flags, uint32(info.Mode()))
if ok && permUnchanged && !cf.IsDeleted() && cf.IsDirectory() && !cf.IsSymlink() && !cf.IsInvalid() {
return nil
}
flags := uint32(protocol.FlagDirectory)
@@ -350,7 +362,7 @@ func (w *Walker) walkDir(relPath string, info os.FileInfo, mtime time.Time, dcha
}
f := protocol.FileInfo{
Name: relPath,
Version: currentVersion.Update(w.ShortID),
Version: cf.Version.Update(w.ShortID),
Flags: flags,
Modified: mtime.Unix(),
}
@@ -370,7 +382,7 @@ func (w *Walker) walkDir(relPath string, info os.FileInfo, mtime time.Time, dcha
// transcend into symlinks at all, but there are rumours that this may have
// happened anyway under some circumstances, possibly Windows reparse points
// or something. Hence the "skip" return from this one.
func (w *Walker) walkSymlink(absPath, relPath string, dchan chan protocol.FileInfo) (skip bool, err error) {
func (w *walker) walkSymlink(absPath, relPath string, dchan chan protocol.FileInfo) (skip bool, err error) {
// If the target is a directory, do NOT descend down there. This will
// cause files to get tracked, and removing the symlink will as a result
// remove files in their real location.
@@ -395,25 +407,21 @@ func (w *Walker) walkSymlink(absPath, relPath string, dchan chan protocol.FileIn
return true, nil
}
var currentVersion protocol.Vector
if w.CurrentFiler != nil {
// A symlink is "unchanged", if
// - it exists
// - it wasn't deleted (because it isn't now)
// - it was a symlink
// - it wasn't invalid
// - the symlink type (file/dir) was the same
// - the block list (i.e. hash of target) was the same
cf, ok := w.CurrentFiler.CurrentFile(relPath)
if ok && !cf.IsDeleted() && cf.IsSymlink() && !cf.IsInvalid() && SymlinkTypeEqual(targetType, cf) && BlocksEqual(cf.Blocks, blocks) {
return true, nil
}
currentVersion = cf.Version
// A symlink is "unchanged", if
// - it exists
// - it wasn't deleted (because it isn't now)
// - it was a symlink
// - it wasn't invalid
// - the symlink type (file/dir) was the same
// - the block list (i.e. hash of target) was the same
cf, ok := w.CurrentFiler.CurrentFile(relPath)
if ok && !cf.IsDeleted() && cf.IsSymlink() && !cf.IsInvalid() && SymlinkTypeEqual(targetType, cf) && BlocksEqual(cf.Blocks, blocks) {
return true, nil
}
f := protocol.FileInfo{
Name: relPath,
Version: currentVersion.Update(w.ShortID),
Version: cf.Version.Update(w.ShortID),
Flags: uint32(protocol.FlagSymlink | protocol.FlagNoPermBits | 0666 | SymlinkFlags(targetType)),
Modified: 0,
Blocks: blocks,
@@ -432,7 +440,7 @@ func (w *Walker) walkSymlink(absPath, relPath string, dchan chan protocol.FileIn
// normalizePath returns the normalized relative path (possibly after fixing
// it on disk), or skip is true.
func (w *Walker) normalizePath(absPath, relPath string) (normPath string, skip bool) {
func (w *walker) normalizePath(absPath, relPath string) (normPath string, skip bool) {
if runtime.GOOS == "darwin" {
// Mac OS X file names should always be NFD normalized.
normPath = norm.NFD.String(relPath)
@@ -574,3 +582,27 @@ func (c *byteCounter) Total() int64 {
func (c *byteCounter) Close() {
close(c.stop)
}
// A no-op CurrentFiler
type noCurrentFiler struct{}
func (noCurrentFiler) CurrentFile(name string) (protocol.FileInfo, bool) {
return protocol.FileInfo{}, false
}
// A no-op TempNamer
type noTempNamer struct{}
func (noTempNamer) IsTemporary(path string) bool {
return false
}
// A no-op MtimeRepo
type noMtimeRepo struct{}
func (noMtimeRepo) GetMtime(relPath string, mtime time.Time) time.Time {
return mtime
}
+11 -16
View File
@@ -59,14 +59,13 @@ func TestWalkSub(t *testing.T) {
t.Fatal(err)
}
w := Walker{
fchan, err := Walk(Config{
Dir: "testdata",
Subs: []string{"dir2"},
BlockSize: 128 * 1024,
Matcher: ignores,
Hashers: 2,
}
fchan, err := w.Walk()
})
var files []protocol.FileInfo
for f := range fchan {
files = append(files, f)
@@ -97,14 +96,13 @@ func TestWalk(t *testing.T) {
}
t.Log(ignores)
w := Walker{
fchan, err := Walk(Config{
Dir: "testdata",
BlockSize: 128 * 1024,
Matcher: ignores,
Hashers: 2,
}
})
fchan, err := w.Walk()
if err != nil {
t.Fatal(err)
}
@@ -122,22 +120,20 @@ func TestWalk(t *testing.T) {
}
func TestWalkError(t *testing.T) {
w := Walker{
_, err := Walk(Config{
Dir: "testdata-missing",
BlockSize: 128 * 1024,
Hashers: 2,
}
_, err := w.Walk()
})
if err == nil {
t.Error("no error from missing directory")
}
w = Walker{
_, err = Walk(Config{
Dir: "testdata/bar",
BlockSize: 128 * 1024,
}
_, err = w.Walk()
})
if err == nil {
t.Error("no error from non-directory")
@@ -278,7 +274,7 @@ func TestNormalization(t *testing.T) {
}
func TestIssue1507(t *testing.T) {
w := Walker{}
w := &walker{}
c := make(chan protocol.FileInfo, 100)
fn := w.walkAndHashFiles(c, c)
@@ -286,14 +282,13 @@ func TestIssue1507(t *testing.T) {
}
func walkDir(dir string) ([]protocol.FileInfo, error) {
w := Walker{
fchan, err := Walk(Config{
Dir: dir,
BlockSize: 128 * 1024,
AutoNormalize: true,
Hashers: 2,
}
})
fchan, err := w.Walk()
if err != nil {
return nil, err
}
+7 -2
View File
@@ -10,6 +10,7 @@ package upgrade
import (
"errors"
"fmt"
"path"
"runtime"
"strconv"
"strings"
@@ -63,12 +64,12 @@ func To(rel Release) error {
func ToURL(url string) error {
select {
case <-upgradeUnlocked:
path, err := osext.Executable()
binary, err := osext.Executable()
if err != nil {
upgradeUnlocked <- true
return err
}
err = upgradeToURL(path, url)
err = upgradeToURL(path.Base(url), binary, url)
// If we've failed to upgrade, unlock so that another attempt could be made
if err != nil {
upgradeUnlocked <- true
@@ -219,6 +220,10 @@ func versionParts(v string) ([]int, []interface{}) {
}
func releaseName(tag string) string {
// We must ensure that the release asset matches the expected naming
// standard, containing both the architecture/OS and the tag name we
// expect. This protects against malformed release data potentially
// tricking us into doing a downgrade.
switch runtime.GOOS {
case "darwin":
return fmt.Sprintf("syncthing-macosx-%s-%s.", runtime.GOARCH, tag)
+99 -22
View File
@@ -25,6 +25,7 @@ import (
"runtime"
"sort"
"strings"
"time"
"github.com/syncthing/syncthing/lib/dialer"
"github.com/syncthing/syncthing/lib/signature"
@@ -32,12 +33,38 @@ import (
const DisabledByCompilation = false
const (
// Current binary size hovers around 10 MB. We give it some room to grow
// and say that we never expect the binary to be larger than 64 MB.
maxBinarySize = 64 << 20 // 64 MiB
// The max expected size of the signature file.
maxSignatureSize = 10 << 10 // 10 KiB
// We set the same limit on the archive. The binary will compress and we
// include some other stuff - currently the release archive size is
// around 6 MB.
maxArchiveSize = maxBinarySize
// When looking through the archive for the binary and signature, stop
// looking once we've searched this many files.
maxArchiveMembers = 100
// Archive reads, or metadata checks, that take longer than this will be
// rejected.
readTimeout = 30 * time.Minute
// The limit on the size of metadata that we accept.
maxMetadataSize = 10 << 20 // 10 MiB
)
// This is an HTTP/HTTPS client that does *not* perform certificate
// validation. We do this because some systems where Syncthing runs have
// issues with old or missing CA roots. It doesn't actually matter that we
// load the upgrade insecurely as we verify an ECDSA signature of the actual
// binary contents before accepting the upgrade.
var insecureHTTP = &http.Client{
Timeout: readTimeout,
Transport: &http.Transport{
Dial: dialer.Dial,
Proxy: http.ProxyFromEnvironment,
@@ -47,10 +74,20 @@ var insecureHTTP = &http.Client{
},
}
func insecureGet(url, version string) (*http.Response, error) {
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", fmt.Sprintf(`syncthing %s (%s %s-%s)`, version, runtime.Version(), runtime.GOOS, runtime.GOARCH))
return insecureHTTP.Do(req)
}
// FetchLatestReleases returns the latest releases, including prereleases or
// not depending on the argument
func FetchLatestReleases(releasesURL, version string) []Release {
resp, err := insecureHTTP.Get(releasesURL)
resp, err := insecureGet(releasesURL, version)
if err != nil {
l.Infoln("Couldn't fetch release information:", err)
return nil
@@ -61,7 +98,10 @@ func FetchLatestReleases(releasesURL, version string) []Release {
}
var rels []Release
json.NewDecoder(resp.Body).Decode(&rels)
err = json.NewDecoder(io.LimitReader(resp.Body, maxMetadataSize)).Decode(&rels)
if err != nil {
l.Infoln("Fetching release information:", err)
}
resp.Body.Close()
return rels
@@ -120,7 +160,7 @@ func upgradeTo(binary string, rel Release) error {
l.Debugln("considering release", assetName)
if strings.HasPrefix(assetName, expectedRelease) {
return upgradeToURL(binary, asset.URL)
return upgradeToURL(assetName, binary, asset.URL)
}
}
@@ -128,8 +168,8 @@ func upgradeTo(binary string, rel Release) error {
}
// Upgrade to the given release, saving the previous binary with a ".old" extension.
func upgradeToURL(binary string, url string) error {
fname, err := readRelease(filepath.Dir(binary), url)
func upgradeToURL(archiveName, binary string, url string) error {
fname, err := readRelease(archiveName, filepath.Dir(binary), url)
if err != nil {
return err
}
@@ -147,7 +187,7 @@ func upgradeToURL(binary string, url string) error {
return nil
}
func readRelease(dir, url string) (string, error) {
func readRelease(archiveName, dir, url string) (string, error) {
l.Debugf("loading %q", url)
req, err := http.NewRequest("GET", url, nil)
@@ -164,13 +204,13 @@ func readRelease(dir, url string) (string, error) {
switch runtime.GOOS {
case "windows":
return readZip(dir, resp.Body)
return readZip(archiveName, dir, io.LimitReader(resp.Body, maxArchiveSize))
default:
return readTarGz(dir, resp.Body)
return readTarGz(archiveName, dir, io.LimitReader(resp.Body, maxArchiveSize))
}
}
func readTarGz(dir string, r io.Reader) (string, error) {
func readTarGz(archiveName, dir string, r io.Reader) (string, error) {
gr, err := gzip.NewReader(r)
if err != nil {
return "", err
@@ -182,7 +222,13 @@ func readTarGz(dir string, r io.Reader) (string, error) {
var sig []byte
// Iterate through the files in the archive.
i := 0
for {
if i >= maxArchiveMembers {
break
}
i++
hdr, err := tr.Next()
if err == io.EOF {
// end of tar archive
@@ -191,6 +237,11 @@ func readTarGz(dir string, r io.Reader) (string, error) {
if err != nil {
return "", err
}
if hdr.Size > maxBinarySize {
// We don't even want to try processing or skipping over files
// that are too large.
break
}
err = archiveFileVisitor(dir, &tempName, &sig, hdr.Name, tr)
if err != nil {
@@ -202,14 +253,14 @@ func readTarGz(dir string, r io.Reader) (string, error) {
}
}
if err := verifyUpgrade(tempName, sig); err != nil {
if err := verifyUpgrade(archiveName, tempName, sig); err != nil {
return "", err
}
return tempName, nil
}
func readZip(dir string, r io.Reader) (string, error) {
func readZip(archiveName, dir string, r io.Reader) (string, error) {
body, err := ioutil.ReadAll(r)
if err != nil {
return "", err
@@ -224,7 +275,19 @@ func readZip(dir string, r io.Reader) (string, error) {
var sig []byte
// Iterate through the files in the archive.
i := 0
for _, file := range archive.File {
if i >= maxArchiveMembers {
break
}
i++
if file.UncompressedSize64 > maxBinarySize {
// We don't even want to try processing or skipping over files
// that are too large.
break
}
inFile, err := file.Open()
if err != nil {
return "", err
@@ -241,7 +304,7 @@ func readZip(dir string, r io.Reader) (string, error) {
}
}
if err := verifyUpgrade(tempName, sig); err != nil {
if err := verifyUpgrade(archiveName, tempName, sig); err != nil {
return "", err
}
@@ -254,23 +317,24 @@ func archiveFileVisitor(dir string, tempFile *string, signature *[]byte, archive
var err error
filename := path.Base(archivePath)
archiveDir := path.Dir(archivePath)
archiveDirs := strings.Split(archiveDir, "/")
if len(archiveDirs) > 1 {
//don't consider files in subfolders
return nil
}
l.Debugf("considering file %s", archivePath)
switch filename {
case "syncthing", "syncthing.exe":
archiveDirs := strings.Split(archiveDir, "/")
if len(archiveDirs) > 1 {
// Don't consider "syncthing" files found too deeply, as they may be
// other things.
return nil
}
l.Debugf("found upgrade binary %s", archivePath)
*tempFile, err = writeBinary(dir, filedata)
*tempFile, err = writeBinary(dir, io.LimitReader(filedata, maxBinarySize))
if err != nil {
return err
}
case "syncthing.sig", "syncthing.exe.sig":
case "release.sig":
l.Debugf("found signature %s", archivePath)
*signature, err = ioutil.ReadAll(filedata)
*signature, err = ioutil.ReadAll(io.LimitReader(filedata, maxSignatureSize))
if err != nil {
return err
}
@@ -279,7 +343,7 @@ func archiveFileVisitor(dir string, tempFile *string, signature *[]byte, archive
return nil
}
func verifyUpgrade(tempName string, sig []byte) error {
func verifyUpgrade(archiveName, tempName string, sig []byte) error {
if tempName == "" {
return fmt.Errorf("no upgrade found")
}
@@ -293,7 +357,20 @@ func verifyUpgrade(tempName string, sig []byte) error {
if err != nil {
return err
}
err = signature.Verify(SigningKey, sig, fd)
// Create a new reader that will serve reads from, in order:
//
// - the archive name ("syncthing-linux-amd64-v0.13.0-beta.4.tar.gz")
// followed by a newline
//
// - the temp file contents
//
// We then verify the release signature against the contents of this
// multireader. This ensures that it is not only a bonafide syncthing
// binary, but it it also of exactly the platform and version we expect.
mr := io.MultiReader(bytes.NewBufferString(archiveName+"\n"), fd)
err = signature.Verify(SigningKey, sig, mr)
fd.Close()
if err != nil {
+1 -1
View File
@@ -14,7 +14,7 @@ func upgradeTo(binary string, rel Release) error {
return ErrUpgradeUnsupported
}
func upgradeToURL(binary, url string) error {
func upgradeToURL(archiveName, binary, url string) error {
return ErrUpgradeUnsupported
}
+6 -1
View File
@@ -15,7 +15,12 @@ import (
)
// randomCharset contains the characters that can make up a randomString().
const randomCharset = "01234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-"
const randomCharset = "2345679abcdefghijkmnopqrstuvwxyzACDEFGHJKLMNPQRSTUVWXYZ"
func init() {
// The default RNG should be seeded with something good.
mathRand.Seed(RandomInt64())
}
// RandomString returns a string of random characters (taken from
// randomCharset) of the specified length.
+3 -3
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-BEP" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-BEP" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-bep \- Block Exchange Protocol v1
.
@@ -232,7 +232,7 @@ For C=1:
The Length field contains the length, in bytes, of the compressed
message data plus a four byte uncompressed length field.
.IP \(bu 2
The compressed message data is preceeded by a 32 bit field denoting
The compressed message data is preceded by a 32 bit field denoting
the length of the uncompressed message.
.IP \(bu 2
The message data is compressed using the LZ4 format and algorithm
@@ -1149,7 +1149,7 @@ is no longer available, therefore the list of block indexes should be truncated.
Messages with \fBForget\fP bit set MUST NOT have any block indexes.
.sp
Any update message which is being sent for a different \fBVersion\fP of the same
file name must be preceeded with an update message for the old version of that
file name must be preceded with an update message for the old version of that
file with the \fBForget\fP bit set.
.sp
As a safeguard on the receiving side, value of \fBVersion\fP changing between
+125 -51
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-CONFIG" "5" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-CONFIG" "5" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-config \- Syncthing Configuration
.
@@ -81,8 +81,8 @@ The following shows the default configuration file:
.sp
.nf
.ft C
<configuration version="12">
<folder id="default" path="/Users/jb/Sync/" ro="false" rescanIntervalS="60" ignorePerms="false" autoNormalize="true">
<configuration version="14">
<folder id="zj2AA\-q55a7" label="Default Folder (zj2AA\-q55a7)" path="/Users/jb/Sync/" type="readwrite" rescanIntervalS="60" ignorePerms="false" autoNormalize="true">
<device id="3LT2GA5\-CQI4XJM\-WTZ264P\-MLOGMHL\-MCRLDNT\-MZV4RD3\-KA745CL\-OGAERQZ"></device>
<minDiskFreePct>1</minDiskFreePct>
<versioning></versioning>
@@ -94,34 +94,35 @@ The following shows the default configuration file:
<scanProgressIntervalS>0</scanProgressIntervalS>
<pullerSleepS>0</pullerSleepS>
<pullerPauseS>0</pullerPauseS>
<maxConflicts>0</maxConflicts>
<maxConflicts>\-1</maxConflicts>
<disableSparseFiles>false</disableSparseFiles>
<disableTempIndexes>false</disableTempIndexes>
</folder>
<device id="3LT2GA5\-CQI4XJM\-WTZ264P\-MLOGMHL\-MCRLDNT\-MZV4RD3\-KA745CL\-OGAERQZ" name="syno" compression="metadata" introducer="false">
<address>dynamic</address>
</device>
<gui enabled="true" tls="false">
<address>127.0.0.1:52620</address>
<address>127.0.0.1:8384</address>
<apikey>k1dnz1Dd0rzTBjjFFh7CXPnrF12C49B1</apikey>
<theme>default</theme>
</gui>
<options>
<listenAddress>tcp://0.0.0.0:22000</listenAddress>
<listenAddress>default</listenAddress>
<globalAnnounceServer>default</globalAnnounceServer>
<globalAnnounceEnabled>true</globalAnnounceEnabled>
<localAnnounceEnabled>true</localAnnounceEnabled>
<localAnnouncePort>21027</localAnnouncePort>
<localAnnounceMCAddr>[ff12::8384]:21027</localAnnounceMCAddr>
<relayServer>dynamic+https://relays.syncthing.net/endpoint</relayServer>
<maxSendKbps>0</maxSendKbps>
<maxRecvKbps>0</maxRecvKbps>
<reconnectionIntervalS>60</reconnectionIntervalS>
<relaysEnabled>true</relaysEnabled>
<relayReconnectIntervalM>10</relayReconnectIntervalM>
<relayWithoutGlobalAnn>false</relayWithoutGlobalAnn>
<startBrowser>true</startBrowser>
<upnpEnabled>true</upnpEnabled>
<upnpLeaseMinutes>60</upnpLeaseMinutes>
<upnpRenewalMinutes>30</upnpRenewalMinutes>
<upnpTimeoutSeconds>10</upnpTimeoutSeconds>
<natEnabled>true</natEnabled>
<natLeaseMinutes>60</natLeaseMinutes>
<natRenewalMinutes>30</natRenewalMinutes>
<natTimeoutSeconds>10</natTimeoutSeconds>
<urAccepted>0</urAccepted>
<urUniqueID></urUniqueID>
<urURL>https://data.syncthing.net/newdata</urURL>
@@ -130,13 +131,14 @@ The following shows the default configuration file:
<restartOnWakeup>true</restartOnWakeup>
<autoUpgradeIntervalH>12</autoUpgradeIntervalH>
<keepTemporariesH>24</keepTemporariesH>
<cacheIgnoredFiles>true</cacheIgnoredFiles>
<cacheIgnoredFiles>false</cacheIgnoredFiles>
<progressUpdateIntervalS>5</progressUpdateIntervalS>
<symlinksEnabled>true</symlinksEnabled>
<limitBandwidthInLan>false</limitBandwidthInLan>
<databaseBlockCacheMiB>0</databaseBlockCacheMiB>
<minHomeDiskFreePct>1</minHomeDiskFreePct>
<releasesURL>https://api.github.com/repos/syncthing/syncthing/releases?per_page=30</releasesURL>
<overwriteRemoteDeviceNamesOnConnect>false</overwriteRemoteDeviceNamesOnConnect>
<tempIndexMinBlocks>10</tempIndexMinBlocks>
</options>
</configuration>
.ft P
@@ -158,7 +160,7 @@ migration from previous formats.
.sp
.nf
.ft C
<folder id="default" path="/Users/jb/Sync/" ro="false" rescanIntervalS="60" ignorePerms="false" autoNormalize="true">
<folder id="zj2AA\-q55a7" label="Default Folder (zj2AA\-q55a7)" path="/Users/jb/Sync/" type="readwrite" rescanIntervalS="60" ignorePerms="false" autoNormalize="true" ro="false">
<device id="3LT2GA5\-CQI4XJM\-WTZ264P\-MLOGMHL\-MCRLDNT\-MZV4RD3\-KA745CL\-OGAERQZ"></device>
<minDiskFreePct>1</minDiskFreePct>
<versioning></versioning>
@@ -170,7 +172,9 @@ migration from previous formats.
<scanProgressIntervalS>0</scanProgressIntervalS>
<pullerSleepS>0</pullerSleepS>
<pullerPauseS>0</pullerPauseS>
<maxConflicts>0</maxConflicts>
<maxConflicts>\-1</maxConflicts>
<disableSparseFiles>false</disableSparseFiles>
<disableTempIndexes>false</disableTempIndexes>
</folder>
.ft P
.fi
@@ -185,13 +189,25 @@ element:
.B id
The folder ID, must be unique. (mandatory)
.TP
.B label
The label of a folder is a human readable and descriptive local name.
Can be different on each device. (optional)
.TP
.B path
The path to the directory where the folder is stored on this
device; not sent to other devices. (mandatory)
.TP
.B ro
True if the folder is read only (Master mode; will not be modified by
Syncthing) on this device.
.B type
Controls how the folder is handled by Syncthing. Possible values are:
.INDENT 7.0
.TP
.B readwrite
The folder is in default mode. Sending local and accepting remote changes.
.TP
.B readonly
The folder is in "master" mode \-\- it will not be modified by
syncthing on this device.
.UNINDENT
.TP
.B rescanIntervalS
The rescan interval, in seconds. Can be set to zero to disable when external
@@ -274,6 +290,16 @@ what you\(aqre doing.
The maximum number of conflict copies to keep around for any given file.
The default, \-1, means an unlimited number. Setting this to zero disables
conflict copies altogether.
.TP
.B disableSparseFiles
By default, blocks containing all zeroes are not written, causing files
to be sparse on filesystems that support the concept. When set to true,
sparse files will not be created.
.TP
.B disableTempIndexes
By default, devices exchange information about blocks available in
transfers that are still in progress. When set to true, such information
is not exchanged for this folder.
.UNINDENT
.SH DEVICE ELEMENT
.INDENT 0.0
@@ -384,6 +410,7 @@ This optional element lists device IDs that have been specifically ignored. One
<gui enabled="true" tls="false">
<address>127.0.0.1:8384</address>
<apikey>l7jSbCqPD95JYZ0g8vi4ZLAMg3ulnN1b</apikey>
<theme>default</theme>
</gui>
.ft P
.fi
@@ -402,6 +429,9 @@ If not \fBtrue\fP, the GUI and API will not be started.
If set to \fBtrue\fP, TLS (HTTPS) will be enforced. Non\-HTTPS requests will
be redirected to HTTPS. When this is set to \fBfalse\fP, TLS connections are
still possible but it is not mandatory.
.TP
.B theme
The name of the theme to use.
.UNINDENT
.sp
The following child elements may be present:
@@ -415,16 +445,10 @@ Allowed address formats are:
.B IPv4 address and port (\fB127.0.0.1:8384\fP)
The address and port is used as given.
.TP
.B IPv4 wildcard and port (\fBtcp4://0.0.0.0\fP, \fBtcp4://:8384\fP)
These are equivalent and will result in Syncthing listening on all interfaces via IPv4 only.
.TP
.B IPv6 address and port (\fB[::1]:8384\fP)
The address and port is used as given. The address must be enclosed in
square brackets.
.TP
.B IPv6 wildcard and port (\fBtcp6://[::]:8384\fP, \fBtcp6://:8384\fP)
These are equivalent and will result in Syncthing listening on all interfaces via IPv6 only.
.TP
.B Wildcard and port (\fB0.0.0.0:12345\fP, \fB[::]:12345\fP, \fB:12345\fP)
These are equivalent and will result in Syncthing listening on all
interfaces via both IPv4 and IPv6.
@@ -446,24 +470,22 @@ If set, this is the API key that enables usage of the REST interface.
.nf
.ft C
<options>
<listenAddress>tcp://0.0.0.0:22000</listenAddress>
<listenAddress>default</listenAddress>
<globalAnnounceServer>default</globalAnnounceServer>
<globalAnnounceEnabled>true</globalAnnounceEnabled>
<localAnnounceEnabled>true</localAnnounceEnabled>
<localAnnouncePort>21027</localAnnouncePort>
<localAnnounceMCAddr>[ff12::8384]:21027</localAnnounceMCAddr>
<relayServer>dynamic+https://relays.syncthing.net/endpoint</relayServer>
<maxSendKbps>0</maxSendKbps>
<maxRecvKbps>0</maxRecvKbps>
<reconnectionIntervalS>60</reconnectionIntervalS>
<relaysEnabled>true</relaysEnabled>
<relayReconnectIntervalM>10</relayReconnectIntervalM>
<relayWithoutGlobalAnn>false</relayWithoutGlobalAnn>
<startBrowser>true</startBrowser>
<upnpEnabled>true</upnpEnabled>
<upnpLeaseMinutes>60</upnpLeaseMinutes>
<upnpRenewalMinutes>30</upnpRenewalMinutes>
<upnpTimeoutSeconds>10</upnpTimeoutSeconds>
<natEnabled>true</natEnabled>
<natLeaseMinutes>60</natLeaseMinutes>
<natRenewalMinutes>30</natRenewalMinutes>
<natTimeoutSeconds>10</natTimeoutSeconds>
<urAccepted>0</urAccepted>
<urUniqueID></urUniqueID>
<urURL>https://data.syncthing.net/newdata</urURL>
@@ -472,13 +494,14 @@ If set, this is the API key that enables usage of the REST interface.
<restartOnWakeup>true</restartOnWakeup>
<autoUpgradeIntervalH>12</autoUpgradeIntervalH>
<keepTemporariesH>24</keepTemporariesH>
<cacheIgnoredFiles>true</cacheIgnoredFiles>
<cacheIgnoredFiles>false</cacheIgnoredFiles>
<progressUpdateIntervalS>5</progressUpdateIntervalS>
<symlinksEnabled>true</symlinksEnabled>
<limitBandwidthInLan>false</limitBandwidthInLan>
<databaseBlockCacheMiB>0</databaseBlockCacheMiB>
<minHomeDiskFreePct>1</minHomeDiskFreePct>
<releasesURL>https://api.github.com/repos/syncthing/syncthing/releases?per_page=30</releasesURL>
<overwriteRemoteDeviceNamesOnConnect>false</overwriteRemoteDeviceNamesOnConnect>
<tempIndexMinBlocks>10</tempIndexMinBlocks>
</options>
.ft P
.fi
@@ -489,10 +512,8 @@ The \fBoptions\fP element contains all other global configuration options.
.INDENT 0.0
.TP
.B listenAddress
The listen address for incoming sync connections. See the \fBaddress\fP
element under the \fI\%GUI Element\fP for allowed syntax, with the addition
that the address must have a protocol scheme prefix. Currently \fBtcp://\fP
is the only supported protocol scheme.
The listen address for incoming sync connections. See
\fI\%Listen Addresses\fP for allowed syntax.
.TP
.B globalAnnounceServer
A URI to a global announce (discovery) server, or the word \fBdefault\fP to
@@ -539,25 +560,20 @@ When true, relays will be connected to and potentially used for device to device
.B relayReconnectIntervalM
Sets the interval, in minutes, between relay reconnect attempts.
.TP
.B relayWithoutGlobalAnn
When set to true, relay connections will be attempted even when global
discovery is disabled. This is useful only in the case where devices are
known to be connected to the same relays. The default is \fBfalse\fP\&.
.TP
.B startBrowser
Whether to attempt to start a browser to show the GUI when Syncthing starts.
.TP
.B upnpEnabled
Whether to attempt to perform an UPnP port mapping for incoming sync
connections.
.B natEnabled
Whether to attempt to perform an UPnP and NAT\-PMP port mapping for
incoming sync connections.
.TP
.B upnpLeaseMinutes
.B natLeaseMinutes
Request a lease for this many minutes; zero to request a permanent lease.
.TP
.B upnpRenewalMinutes
.B natRenewalMinutes
Attempt to renew the lease after this many minutes.
.TP
.B upnpTimeoutSeconds
.B natTimeoutSeconds
When scanning for UPnP devices, wait this long for responses.
.TP
.B urAccepted
@@ -594,8 +610,9 @@ Keep temporary failed transfers for this many hours. While the temporaries
are kept, the data they contain need not be transferred again.
.TP
.B cacheIgnoredFiles
Whether to cache the results of ignore pattern evaluation. Performance at
the price of memory.
Whether to cache the results of ignore pattern evaluation. Performance
at the price of memory. Defaults to \fBfalse\fP as the cost for evaluating
ignores is usually not significant.
.TP
.B progressUpdateIntervalS
How often in seconds the progress of ongoing downloads is made available to
@@ -626,6 +643,63 @@ the configuration and index.
.TP
.B releasesURL
The URL from which release information is loaded, for automatic upgrades.
.TP
.B overwriteRemoteDeviceNamesOnConnect
If set, device names will always be overwritten with the name given by
remote on each connection. By default, the name that the remote device
announces will only be adopted when a name has not already been set.
.TP
.B tempIndexMinBlocks
When exchanging index information for incomplete transfers, only take
into account files that have at least this many blocks.
.UNINDENT
.SS Listen Addresses
.sp
The following address types are accepted in sync protocol listen addresses:
.INDENT 0.0
.TP
.B TCP wildcard and port (\fBtcp://0.0.0.0:22000\fP, \fBtcp://:22000\fP)
These are equivalent and will result in Syncthing listening on all
interfaces, IPv4 and IPv6, on the specified port.
.TP
.B TCP IPv4 wildcard and port (\fBtcp4://0.0.0.0:22000\fP, \fBtcp4://:22000\fP)
These are equivalent and will result in Syncthing listening on all
interfaces via IPv4 only.
.TP
.B TCP IPv4 address and port (\fBtcp4://192.0.2.1:22000\fP)
These are equivalent and will result in Syncthing listening on the
specified address and port only.
.TP
.B TCP IPv6 wildcard and port (\fBtcp6://[::]:22000\fP, \fBtcp6://:22000\fP)
These are equivalent and will result in Syncthing listening on all
interfaces via IPv6 only.
.TP
.B TCP IPv6 address and port (\fBtcp6://[2001:db8::42]:22000\fP)
These are equivalent and will result in Syncthing listening on the
specified address and port only.
.TP
.B Static relay address (\fBrelay://192.0.2.42:22067?id=abcd123...\fP)
Syncthing will connect to and listen for incoming connections via the
specified relay address.
.INDENT 7.0
.INDENT 3.5
.SS Todo
.sp
Document available URL parameters.
.UNINDENT
.UNINDENT
.TP
.B Dynamic relay pool (\fBdynamic+https://192.0.2.42/relays\fP)
Syncthing will fetch the specified HTTPS URL, parse it for a JSON payload
describing relays, select a relay from the available ones and listen via
that as if specified as a static relay above.
.INDENT 7.0
.INDENT 3.5
.SS Todo
.sp
Document available URL parameters.
.UNINDENT
.UNINDENT
.UNINDENT
.SH SYNCING CONFIGURATION FILES
.sp
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-DEVICE-IDS" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-DEVICE-IDS" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-device-ids \- Understanding Device IDs
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-EVENT-API" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-EVENT-API" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-event-api \- Event API
.
+10 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-FAQ" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-FAQ" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-faq \- Frequently Asked Questions
.
@@ -152,6 +152,11 @@ encrypted using AES\-128. When receiving data, it must be decrypted.
.IP 3. 3
There is a certain amount of housekeeping that must be done to track the
current and available versions of each file in the index database.
.IP 4. 3
By default Syncthing uses periodic scanning every 60 seconds to detect
file changes. This means checking every file\(aqs modification time and
comparing it to the database. This can cause spikes of CPU usage for large
folders.
.UNINDENT
.sp
Hashing, compression and encryption cost CPU time. Also, using the GUI
@@ -164,6 +169,10 @@ environment variable \fBGOMAXPROCS\fP to the maximum number of CPU cores
Syncthing should use at any given moment. For example, \fBGOMAXPROCS=2\fP on a
machine with four cores will limit Syncthing to no more than half the
system\(aqs CPU power.
.sp
To reduce CPU spikes from scanning activity, use a filesystem notifications
plugin. This is delivered by default via Synctrayzor, Syncthing\-GTK and on
Android. For other setups, consider using \fI\%syncthing\-inotify\fP <\fBhttps://github.com/syncthing/syncthing-inotify\fP>\&.
.SS Should I keep my device IDs secret?
.sp
No. The IDs are not sensitive. Given a device ID it\(aqs possible to find the IP
+3 -3
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-GLOBALDISCO" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-GLOBALDISCO" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-globaldisco \- Global Discovery Protocol v3
.
@@ -67,7 +67,7 @@ certificate was presented, status \fB403\fP (Forbidden) is returned. If the
posted data doesn\(aqt conform to the expected format, \fB400\fP (Bad Request) is
returned.
.sp
In successfull responses, the server may return a \fBReannounce\-After\fP header
In successful responses, the server may return a \fBReannounce\-After\fP header
containing the number of seconds after which the client should perform a new
announcement.
.sp
@@ -84,7 +84,7 @@ Queries are performed as HTTPS GET requests to the announce server URL. The
requested device ID is passed as the query parameter "device", in canonical
string form, i.e. \fBhttps://announce.syncthing.net/?device=ABC12345\-....\fP
.sp
Successfull responses will have status code \fB200\fP (OK) and carry a JSON payload
Successful responses will have status code \fB200\fP (OK) and carry a JSON payload
of the same format as the announcement above. The response will not contain
empty or unspecified addresses.
.sp
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-LOCALDISCO" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-LOCALDISCO" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-localdisco \- Local Discovery Protocol v3
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-NETWORKING" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-NETWORKING" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-networking \- Firewall Setup
.
+2 -2
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-RELAY" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-RELAY" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-relay \- Relay Protocol v1
.
@@ -337,7 +337,7 @@ _
.TE
.SH MESSAGES
.sp
All messages are preceeded by a header message. Header message contains the
All messages are preceded by a header message. Header message contains the
magic value 0x9E79BC40, message type integer, and message length.
.sp
\fBWARNING:\fP
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-REST-API" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-REST-API" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-rest-api \- REST API
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-SECURITY" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-SECURITY" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-security \- Security Principles
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING-STIGNORE" "5" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING-STIGNORE" "5" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing-stignore \- Prevent files from being synchronized to other nodes
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "TODO" "7" "May 01, 2016" "v0.12" "Syncthing"
.TH "TODO" "7" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
Todo \- Keep automatic backups of deleted files by other nodes
.
+1 -1
View File
@@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SYNCTHING" "1" "May 01, 2016" "v0.12" "Syncthing"
.TH "SYNCTHING" "1" "May 17, 2016" "v0.12" "Syncthing"
.SH NAME
syncthing \- Syncthing
.
+13
View File
@@ -76,6 +76,10 @@ func actualAuthorEmails(paths ...string) stringSet {
continue
}
if strings.Contains(strings.ToLower(body(hash)), "skip-check: authors") {
continue
}
authors.add(author)
}
@@ -99,6 +103,15 @@ func listedAuthorEmails() stringSet {
return authors
}
func body(hash string) string {
cmd := exec.Command("git", "show", "--pretty=format:%b", "-s", hash)
bs, err := cmd.Output()
if err != nil {
log.Fatal("body:", err)
}
return string(bs)
}
// A simple string set type
type stringSet map[string]struct{}
+22 -13
View File
@@ -1,5 +1,5 @@
<configuration version="12">
<folder id="default" path="s1/" ro="false" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
<configuration version="14">
<folder id="default" label="" path="s1/" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
<device id="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU"></device>
<device id="MRIW7OK-NETT3M4-N6SBWME-N25O76W-YJKVXPH-FUMQJ3S-P57B74J-GBITBAC"></device>
<device id="373HSRP-QLPNLIE-JYKZVQF-P4PKZ63-R2ZE6K3-YD442U2-JHBGBQG-WWXAHAU"></device>
@@ -15,8 +15,10 @@
<pullerSleepS>0</pullerSleepS>
<pullerPauseS>0</pullerPauseS>
<maxConflicts>-1</maxConflicts>
<disableSparseFiles>false</disableSparseFiles>
<disableTempIndexes>false</disableTempIndexes>
</folder>
<folder id="¯\_(ツ)_/¯ Räksmörgås 动作 Адрес" path="s12-1/" ro="false" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
<folder id="¯\_(ツ)_/¯ Räksmörgås 动作 Адрес" label="" path="s12-1/" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
<device id="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU"></device>
<device id="MRIW7OK-NETT3M4-N6SBWME-N25O76W-YJKVXPH-FUMQJ3S-P57B74J-GBITBAC"></device>
<minDiskFreePct>1</minDiskFreePct>
@@ -30,6 +32,8 @@
<pullerSleepS>0</pullerSleepS>
<pullerPauseS>0</pullerPauseS>
<maxConflicts>-1</maxConflicts>
<disableSparseFiles>false</disableSparseFiles>
<disableTempIndexes>false</disableTempIndexes>
</folder>
<device id="EJHMPAQ-OGCVORE-ISB4IS3-SYYVJXF-TKJGLTU-66DIQPF-GJ5D2GX-GQ3OWQK" name="s4" compression="metadata" introducer="false">
<address>tcp://127.0.0.1:22004</address>
@@ -51,26 +55,25 @@
<user>testuser</user>
<password>$2a$10$7tKL5uvLDGn5s2VLPM2yWOK/II45az0mTel8hxAUJDRQN1Tk2QYwu</password>
<apikey>abc123</apikey>
<theme>default</theme>
</gui>
<options>
<listenAddress>tcp://127.0.0.1:22001</listenAddress>
<listenAddress>dynamic+https://relays.syncthing.net/endpoint</listenAddress>
<globalAnnounceServer>default</globalAnnounceServer>
<globalAnnounceEnabled>false</globalAnnounceEnabled>
<localAnnounceEnabled>true</localAnnounceEnabled>
<localAnnouncePort>21027</localAnnouncePort>
<localAnnounceMCAddr>[ff12::8384]:21027</localAnnounceMCAddr>
<relayServer>dynamic+https://relays.syncthing.net/endpoint</relayServer>
<maxSendKbps>0</maxSendKbps>
<maxRecvKbps>0</maxRecvKbps>
<reconnectionIntervalS>5</reconnectionIntervalS>
<relaysEnabled>true</relaysEnabled>
<relayReconnectIntervalM>10</relayReconnectIntervalM>
<relayWithoutGlobalAnn>false</relayWithoutGlobalAnn>
<startBrowser>false</startBrowser>
<upnpEnabled>true</upnpEnabled>
<upnpLeaseMinutes>0</upnpLeaseMinutes>
<upnpRenewalMinutes>30</upnpRenewalMinutes>
<upnpTimeoutSeconds>10</upnpTimeoutSeconds>
<natEnabled>true</natEnabled>
<natLeaseMinutes>0</natLeaseMinutes>
<natRenewalMinutes>30</natRenewalMinutes>
<natTimeoutSeconds>10</natTimeoutSeconds>
<urAccepted>-1</urAccepted>
<urUniqueID></urUniqueID>
<urURL>https://data.syncthing.net/newdata</urURL>
@@ -79,12 +82,18 @@
<restartOnWakeup>true</restartOnWakeup>
<autoUpgradeIntervalH>12</autoUpgradeIntervalH>
<keepTemporariesH>24</keepTemporariesH>
<cacheIgnoredFiles>true</cacheIgnoredFiles>
<cacheIgnoredFiles>false</cacheIgnoredFiles>
<progressUpdateIntervalS>5</progressUpdateIntervalS>
<symlinksEnabled>true</symlinksEnabled>
<limitBandwidthInLan>false</limitBandwidthInLan>
<databaseBlockCacheMiB>0</databaseBlockCacheMiB>
<minHomeDiskFreePct>1</minHomeDiskFreePct>
<releasesURL>https://api.github.com/repos/syncthing/syncthing/releases?per_page=30</releasesURL>
<releasesURL>https://upgrades.syncthing.net/meta.json</releasesURL>
<overwriteRemoteDeviceNamesOnConnect>false</overwriteRemoteDeviceNamesOnConnect>
<tempIndexMinBlocks>10</tempIndexMinBlocks>
<upnpEnabled>true</upnpEnabled>
<upnpLeaseMinutes>0</upnpLeaseMinutes>
<upnpRenewalMinutes>30</upnpRenewalMinutes>
<upnpTimeoutSeconds>10</upnpTimeoutSeconds>
<relaysEnabled>false</relaysEnabled>
</options>
</configuration>
+2 -4
View File
@@ -1,11 +1,9 @@
package natpmp
import (
"testing"
)
import "testing"
func TestNatPMP(t *testing.T) {
client, err := NewClientForDefaultGateway()
client, err := NewClientForDefaultGateway(0)
if err != nil {
t.Errorf("NewClientForDefaultGateway() = %v,%v", client, err)
return
+79 -63
View File
@@ -9,7 +9,7 @@ import (
const (
char_any = '*'
char_separator = ','
char_comma = ','
char_single = '?'
char_escape = '\\'
char_range_open = '['
@@ -138,7 +138,7 @@ type lexer struct {
func newLexer(source string) *lexer {
l := &lexer{
input: source,
state: lexText,
state: lexRaw,
items: make(chan item, len(source)),
termPhrases: make(map[int]int),
}
@@ -146,7 +146,7 @@ func newLexer(source string) *lexer {
}
func (l *lexer) run() {
for state := lexText; state != nil; {
for state := lexRaw; state != nil; {
state = state(l)
}
close(l.items)
@@ -218,29 +218,26 @@ func (l *lexer) acceptAll(valid string) {
l.unread()
}
func (l *lexer) emit(t itemType) {
if l.pos == len(l.input) {
l.items <- item{t, l.input[l.start:]}
} else {
l.items <- item{t, l.input[l.start:l.pos]}
}
func (l *lexer) emitCurrent(t itemType) {
l.emit(t, l.input[l.start:l.pos])
}
func (l *lexer) emit(t itemType, s string) {
l.items <- item{t, s}
l.start = l.pos
l.runes = 0
l.width = 0
}
func (l *lexer) emitMaybe(t itemType) {
if l.pos > l.start {
l.emit(t)
}
}
func (l *lexer) errorf(format string, args ...interface{}) {
l.items <- item{item_error, fmt.Sprintf(format, args...)}
}
func lexText(l *lexer) stateFn {
func (l *lexer) inTerms() bool {
return len(l.termScopes) > 0
}
func lexRaw(l *lexer) stateFn {
for {
c := l.read()
if c == eof {
@@ -248,21 +245,8 @@ func lexText(l *lexer) stateFn {
}
switch c {
case char_escape:
l.unread()
l.emitMaybe(item_text)
l.read()
l.ignore()
if l.read() == eof {
l.errorf("unclosed '%s' character", string(char_escape))
return nil
}
case char_single:
l.unread()
l.emitMaybe(item_text)
return lexSingle
case char_any:
@@ -274,33 +258,35 @@ func lexText(l *lexer) stateFn {
}
l.unread()
l.emitMaybe(item_text)
return n
case char_range_open:
l.unread()
l.emitMaybe(item_text)
return lexRangeOpen
case char_terms_open:
l.unread()
l.emitMaybe(item_text)
return lexTermsOpen
case char_terms_close:
l.unread()
l.emitMaybe(item_text)
return lexTermsClose
case char_separator:
case char_comma:
if l.inTerms() { // if we are not in terms
l.unread()
return lexSeparator
}
fallthrough
default:
l.unread()
l.emitMaybe(item_text)
return lexSeparator
return lexText
}
}
if l.pos > l.start {
l.emit(item_text)
l.emitCurrent(item_text)
}
if len(l.termScopes) != 0 {
@@ -308,11 +294,41 @@ func lexText(l *lexer) stateFn {
return nil
}
l.emit(item_eof)
l.emitCurrent(item_eof)
return nil
}
func lexText(l *lexer) stateFn {
var escaped bool
var data []rune
scan:
for c := l.read(); c != eof; c = l.read() {
switch {
case c == char_escape:
escaped = true
continue
case !escaped && c == char_comma && l.inTerms():
l.unread()
break scan
case !escaped && utf8.RuneLen(c) == 1 && special(byte(c)):
l.unread()
break scan
default:
data = append(data, c)
}
escaped = false
}
l.emit(item_text, string(data))
return lexRaw
}
func lexInsideRange(l *lexer) stateFn {
for {
c := l.read()
@@ -325,7 +341,7 @@ func lexInsideRange(l *lexer) stateFn {
case char_range_not:
// only first char makes sense
if l.pos-l.width == l.start {
l.emit(item_not)
l.emitCurrent(item_not)
}
case char_range_between:
@@ -338,8 +354,13 @@ func lexInsideRange(l *lexer) stateFn {
return lexRangeHiLo
case char_range_close:
if l.runes == 1 {
l.errorf("range should contain at least single char")
return nil
}
l.unread()
l.emitMaybe(item_text)
l.emitCurrent(item_text)
return lexRangeClose
}
}
@@ -362,7 +383,7 @@ func lexRangeHiLo(l *lexer) stateFn {
return nil
}
l.emit(item_range_between)
l.emitCurrent(item_range_between)
case char_range_close:
l.unread()
@@ -372,7 +393,7 @@ func lexRangeHiLo(l *lexer) stateFn {
return nil
}
l.emit(item_range_hi)
l.emitCurrent(item_range_hi)
return lexRangeClose
default:
@@ -385,35 +406,30 @@ func lexRangeHiLo(l *lexer) stateFn {
return nil
}
l.emit(item_range_lo)
l.emitCurrent(item_range_lo)
}
}
}
func lexAny(l *lexer) stateFn {
l.pos += 1
l.emit(item_any)
return lexText
l.emitCurrent(item_any)
return lexRaw
}
func lexSuper(l *lexer) stateFn {
l.pos += 2
l.emit(item_super)
return lexText
l.emitCurrent(item_super)
return lexRaw
}
func lexSingle(l *lexer) stateFn {
l.pos += 1
l.emit(item_single)
return lexText
l.emitCurrent(item_single)
return lexRaw
}
func lexSeparator(l *lexer) stateFn {
if len(l.termScopes) == 0 {
l.errorf("syntax error: separator not inside terms list")
return nil
}
posOpen := l.termScopes[len(l.termScopes)-1]
if l.pos-posOpen == 1 {
@@ -423,16 +439,16 @@ func lexSeparator(l *lexer) stateFn {
l.termPhrases[posOpen] += 1
l.pos += 1
l.emit(item_separator)
return lexText
l.emitCurrent(item_separator)
return lexRaw
}
func lexTermsOpen(l *lexer) stateFn {
l.termScopes = append(l.termScopes, l.pos)
l.pos += 1
l.emit(item_terms_open)
l.emitCurrent(item_terms_open)
return lexText
return lexRaw
}
func lexTermsClose(l *lexer) stateFn {
@@ -460,19 +476,19 @@ func lexTermsClose(l *lexer) stateFn {
delete(l.termPhrases, posOpen)
l.pos += 1
l.emit(item_terms_close)
l.emitCurrent(item_terms_close)
return lexText
return lexRaw
}
func lexRangeOpen(l *lexer) stateFn {
l.pos += 1
l.emit(item_range_open)
l.emitCurrent(item_range_open)
return lexInsideRange
}
func lexRangeClose(l *lexer) stateFn {
l.pos += 1
l.emit(item_range_close)
return lexText
l.emitCurrent(item_range_close)
return lexRaw
}
+23 -4
View File
@@ -16,6 +16,27 @@ func TestLexGood(t *testing.T) {
item{item_eof, ""},
},
},
{
pattern: "hello,world",
items: []item{
item{item_text, "hello,world"},
item{item_eof, ""},
},
},
{
pattern: "hello\\,world",
items: []item{
item{item_text, "hello,world"},
item{item_eof, ""},
},
},
{
pattern: "hello\\{world",
items: []item{
item{item_text, "hello{world"},
item{item_eof, ""},
},
},
{
pattern: "hello?",
items: []item{
@@ -124,12 +145,10 @@ func TestLexGood(t *testing.T) {
for i, exp := range test.items {
act := lexer.nextItem()
if act.t != exp.t {
t.Errorf("#%d wrong %d-th item type: exp: %v; act: %v (%s vs %s)", id, i, exp.t, act.t, exp, act)
break
t.Errorf("#%d %q: wrong %d-th item type: exp: %q; act: %q\n\t(%s vs %s)", id, test.pattern, i, exp.t, act.t, exp, act)
}
if act.s != exp.s {
t.Errorf("#%d wrong %d-th item contents: exp: %q; act: %q (%s vs %s)", id, i, exp.s, act.s, exp, act)
break
t.Errorf("#%d %q: wrong %d-th item contents: exp: %q; act: %q\n\t(%s vs %s)", id, test.pattern, i, exp.s, act.s, exp, act)
}
}
}
+2 -2
View File
@@ -4,7 +4,7 @@
{
"importpath": "github.com/AudriusButkevicius/go-nat-pmp",
"repository": "https://github.com/AudriusButkevicius/go-nat-pmp",
"revision": "88a8019a0eff7e9db55f458230b867f0d7e5d48f",
"revision": "e9d7ecafd6f4cd4f59fc45bb9a47466ce637d0fe",
"branch": "master"
},
{
@@ -40,7 +40,7 @@
{
"importpath": "github.com/gobwas/glob",
"repository": "https://github.com/gobwas/glob",
"revision": "d877f6352135181470c40c73ebb81aefa22115fa",
"revision": "82e8d7da03805cde651f981f9702a4b4d8cf58eb",
"branch": "master"
},
{