We've always, since the introduction of conflicts, had the policy that
deletes lose against any other change, for safety's sake. This is a
problem, however, because it means the sort order of versions is not a
total order.
That is, given two versions `A` and `B` that are currently in conflict,
we will sort them in a given order (let's say `A, B`, so `A < B` for
ordering purposes: we say "A wins over B" or "A is newer than B") and
consider the first in the list the winner. The loser (who has `B` on
disk) will process the conflict at some point and move the file to a
conflict copy and announce `A'` as the resolved conflict. The winner
(with `A` on disk) doesn't do anything.
However, if `A` is deleted the ordering changes. We still have `A < B`
and, of course, `Adel < A` (this is not even a conflict, just linear
order). In most sane systems this would imply the ordering `Adel < A <
B`, however in our case we in fact have `B < Adel` because any version
wins over a deleted one, so there is no logical ordering at all of the
files at this point. `Adel < A < B < Adel ???` In practice the deleted
version may end up at the head or the tail of the list, depending on the
order we do the compares.
Hence, at this point, "whatever" happens and it's not guaranteed to make
any sense. 😬
I propose that we resolve this my simply letting deletes be versions
like anything else and maintain a total ordering based on just version
vectors with the existing tie breakers like always. That means a delete
can win in a conflict situation, and the result should be that the file
is moved to a conflict copy on the losing device. I think this retains
the data safety to almost the same degree as previously, while removing
probably an entire class of strange out of sync bugs...
---
(A potential wrinkle here is that, ideally, we wouldn't even create the
conflict copy when the delete and the losing version represent the same
data -- same as when we handle normal modification conflicts. However,
the deleted FileInfo doesn't carry any information on what the contents
were, so we can't do that right now. A possible future extension would
be to carry the block list hash of the deleted data in the deleted
FileInfo and use that for this purpose, but I don't want to complicate
this PR with that. The block list hash itself also isn't a
protocol-defined thing at the moment, it's something implementation
dependent that we just use locally.)
### Purpose
As discussed on the forum:
https://forum.syncthing.net/t/reviving-nat-pmp-in-v2-x-on-android-14/24554
TL;DR
Android 14+ only lets java code get the gateway IPv4 address which is
used in SyncthingNative’s NAT-PMP feature.
So I’ve added the java code to the wrapper, got the router IP address
and feeded it to SyncthingNative by setting the env var
“FALLBACK_NET_GATEWAY_IPV4”.
This revives the NAT feature:
> [Z36WU] INFO: Detected 1 NAT service
### Testing
Local build and test via Android emulator (AVD 15).
This makes a couple of backwards compatible changes to the
ClusterConfig:
- Remove the `ignore_permissions` and `ignore_delete` booleans which
we've never read or used for anything
- Remove the `disable_temp_indexes` boolean and option entirely. We did
use this one, and about 1% of users have set the option. The only thing
it does is inhibits sending of periodical DownloadProgress messages
while downloading data, which is a minuscule bandwidth optimisation
given that we're already sending data at the time.
- Change the `read_only` boolean (which indicated send-only folders) to
an enum `FolderType`, where the values zero and one match the existing
usage. Again, we don't actually use this value, but I can see that we
might want to and then it makes more sense for it to be more
comprehensive.
- Change the `paused` boolean to an enum `StopReason`, where zero
indicates not stopped and one indicates paused, exactly the same wire
representation as previously but leaves space for additional stop
reasons (errors etc).
### Purpose
Filesystem watcher errors didnt have any whitespace between the share
name and the error message, making it hard to read. A simple colon and
whitespace solves this issue
I lately wanted some photos on my phone, and watched them sync
excrutiatingly slowly. I am used to android being slow, but not that
slow. This restriction caught my eye and I increased it beyond the
limit (didn't spot it at first), and I did see a clear improvement. Of
course as always with such a one-off test, I might also have
hallucinated it, but it seems plausible with the slow thing in android
being some layer between the actual filesystem and apps.
Also increase the max limit, mostly just because I don't see any reason
to restrict it that low - not that I have a particular reason to want
more.
I also changed the xml default to 0: The `prepare` code will change it
to the actual default - no need to change that anymore if we change the
default in the future.
Before:
- Local discovery on Android 10+ is broken. The phone receives local
discovery packets from other devices running Syncthing on the same
network, e.g. a computer. But it doesn't send its own local discovery
packets.
- Startup of the beacon/broadcast.go and beacon/multicast.go "services"
subsequently fail, see the log entries of "service.go" with "2 of 2
failures, backing off".
Root cause:
- Android 10+ restricts determining the network interfaces for privacy
reasons. The interfaces and IP addresses cannot be determined.
- There's a bug in the go "net" library. I can actually get the
interfaces, but the fix was not implemented by the go team.
Workaround:
- The "community" found a workaround by creating a light wrapper around
"net" called "anet" library.
- "anet" adjusts the behaviour on Android 10+ and gets the interfaces
plus their IP addresses, as required by Syncthing.
After:
- By using the "anet" lib, Syncthing is able to get the interface ip
addresses and put them into the "AllAddresses" string array.
- The "AllAddresses" string array is then announced on the local
discovery multicast and broadcast packets, if enabled in Syncthing's
config.
- By correctly getting the interfaces and IP addresses using "anet" in
"beacon/broadcast.go" and "beacon/multicast.go", the services start up
fine again.
Verification:
- I've built "libSyncthingNative.so" with this PR applied for Android
and put it into Syncthing-Fork v1.29.7.5 for testing. My two phones,
Android 10 and Android 15 (arm64-v8a) immediately discovered each other
using local discovery.
- I can see the "sent XX bytes" and "recv XX bytes" on both phones in
the log filtering for "SyncthingNativeCode" :-).
Personal note:
- Please go light on me, and, if it's not demanded too much of your
time, please help me on this. I am no go programmer. Most things you
think are easy or common sense aren't part of my knowledge set. I'd just
like to help and hope we somehow can drive this home together to fix the
problem.
----
ref: https://github.com/Catfriend1/syncthing-android/pull/1501
ref: https://github.com/Catfriend1/syncthing-android/issues/1500
ref: https://github.com/wlynxg/anet/blob/main/interface.go &
https://github.com/wlynxg/anet/blob/main/interface_android.go
With that fix, I can see the broadcast/multicast lines again and my
phone can be discovered by other phones running the Syncthing app which
wasn't possible before on Android 10+.
```
[ET76H] .346892 broadcast.go:107: DEBUG: sent 185 bytes to 192.168.x.255:21027
[ET76H] .347114 multicast.go:86: DEBUG: sent 185 bytes to [ff12::8384]:21027 on wlan0
```
---------
Co-authored-by: Marcus B Spencer <marcus@marcusspencer.us>
Fixes#7403.
Tested by enabling UPnP on the router, and checking on the router page
that the external ports of the UDP mappings match what is shown in the
logs and the internal ports matching the QUIC listening port.
Only Require either matching UID & GID OR matching Names.
If the 2 devices have a different Name => UID mapping, they can never be
totaly equal. Therefore when syncing we try matching the Name and fall
back to the UID. However when scanning for changes we currently require
both the Name & UID to match. This leads to forever having out of sync
files back and forth, or local additions when receive only.
This patch does not change the sending behavoir. It only change what we
decide is equal for exisiting files with mismapped Name => UID,
The added testcases show the change: Test 1,5,6 are the same as current.
Test 2,3 Are what change with this patch (from false to true). Test 4 is
a subset of test 2 they is currently special cased as true, which does
not chnage.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
While it doesn't hurt, it's unnecessary since the big protobuf
modernisation, that also introduced types separate from the generated
ones for internal use. Those fields are already dropped when converting
to the wire in protocol.
This updates our key generation to use Ed25519 keys/certificates for
sync connections. Certificates for browser use remain ECDSA for wider
compatibility.
Ed25519 is more modern and has fewer concerns for the future than the
ECDSA curves we used previously. It is supported from Go 1.13 and
forwards, which is Syncthing 1.3.0 (October 2019).
### Purpose
Locally, on Windows 11, and on the windows-2025 GitHub runner (go 1.23
and 1.24), the `TestCopyRange` test is failing with `The request is not
supported.`
On windows-2022 and windows-2019:
```go
err == syscall.ENOTSUP
```
worked, but on Windows 11 and windows-2025, we need:
```go
errors.Is(err, errors.ErrUnsupported)
```
### Testing
Tested on Windows 11, windows-2019, windows-2022, and
[windows-2025](https://github.com/rasa/syncthing/actions/runs/15525123437/job/43703630634#step:7:2811).
* v2: (62 commits)
build: add dependency for next-version script
docs: add release note mention of platforms no longer built
build: streamline gathering of facts, checkouts (#10158)
build: more resilient pushes to releases
chore(etc): add option dash to upstart config
chore(fs): linter complaints
chore(model): the easier linter complaints
chore(internal): linter complaints
chore(sqlite): linter complaints
build: allow v2 into APT candidate channel
docs: link to Docker image, APT, in release notes
build: refactor builds for forks/PRs
build: use same GitHub token for releases as for translation etc pushes
refactor(sqlite): move deleted flag into logical order in schema
feat(config): enable multiple connections by default (#10151)
docs: mention subcommands in release notes, use for all 2.0 releases
docs: adjust release notes for v2.0.0
docs: add relnotes for v2.0.0
build: upgrade setup-zig action (#10134)
fix(versioner): correct fs creation in test
...
* main:
docs: link to Docker image, APT, in release notes
build: also create relaysrv and discosrv releases
fix(stupgrades): return latest stable & pre for each major
fix(syncthing): avoid writing panic log to nil fd (#10154)
### Purpose
This change fixes a logical bug in the panic log writing where we could
end up writing to a uninitialized file descriptor.
On the very first iteration, `panicFd` is nil. We enter the if `panicFd
== nil { … }` block, check for “panic:” or “fatal error:”, and if
neither matches, we skip instantiating `panicFd` altogether. However,
immediately after, still within `if panicFd == nil { … }`, we call
`panicFd.WriteString("Panic at ...")`. But `panicFd` would in this case
be `nil`, which will cause a run‐time panic.
It's not clear to me why panicFd is only initialized if the lines start
with "panic:" or "fatal error:" so I've left that logic untouched. With
this change we at least avoid the risk of writing to a nil
filedescriptor.
## Authorship
Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.
---------
Co-authored-by: Jakob Borg <jakob@kastelo.net>
* main:
feat(config): expose folder and device info as metrics (fixes#9519) (#10148)
chore: add issue types to GitHub issue templates
build: remove schedule from PR metadata job
chore(protocol): only allow enc. password changes on cluster config (#10145)
chore(protocol): don't start connection routines a second time (#10146)
Tihs makes it easier to use metrics based on device and folder labels,
names, and other attributes. Other metrics which are based on folder or
device ID can be joined with these info metrics to enrich their label
sets.
```
# HELP syncthing_config_device_info Provides additional information labels on devices
# TYPE syncthing_config_device_info gauge
syncthing_config_device_info{device="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU",introducer="false",name="s1",paused="false",untrusted="false"} 1
# HELP syncthing_config_folder_info Provides additional information labels on folders
# TYPE syncthing_config_folder_info gauge
syncthing_config_folder_info{folder="default",label="The default folder",path="s2",paused="false",type="sendreceive"} 1
```
With this you can e.g. query for
```
syncthing_connections_active * on(device) group_left syncthing_config_device_info
```
Fixes#9519Closes#10074Closes#10147
This changes the default number of connections from one to three (one
metadata + two data connections). This should give some advantages of
multiple connections, while also not being an overwhelming change for
larger installations. (Though those may need to tweak their settings
anyway, as always.)
In practice we already always call SetPassword and ClusterConfig
together. However it's not just "sensible" to do that, it's required: If
the passwords change, the remote device needs to know about that to
check that the enc. setup is valid/consistent (e.g. tokens match,
folder-type is appropriate, ...).
And with the passwords set later, there's no point in adding them as
part of creating a new connection.
This is a "followup" (if one can call it that 4 years later :) ) to
resp. fix for the following commit:
924b96856f
Co-authored-by: Jakob Borg <jakob@kastelo.net>
This adds a file that will be prepended to release notes (tag messages,
GitHub releases, forum posts) for v1 releases. I'd like there to be
something there to flag that things are going to change.
* main:
refactor: use slices package for sorting (#10136)
build: handle multiple general release notes
build: no need to build on the branches that just trigger tags
* main:
build: use specific token for pushing release tags
fix(gui): update `uncamel()` to handle strings like 'IDs' (fixes#10128) (#10131)
refactor: use slices package for sort (#10132)
build: process for automatic release tags (#10133)
chore(gui, man, authors): update docs, translations, and contributors
> ⚠️ resubmission targeting `main` instead of `v2`
### Purpose
Updates `uncamel()` function in
[uncamelFilter.js](https://github.com/syncthing/syncthing/blob/v2/gui/default/syncthing/core/uncamelFilter.js)
to fix camelCase conversion edge cases, see #10128
This adds an array called `reservedStrings` which will be printed as-is,
e.g. `IDs`, `LAN` etc. I pre-populated this with what I believe makes
sense, but of course this is easily updated.
### Testing
I compiled all the config variables I could find in
`syncthing/lib/config/*configuration.go` and tested this new function
against them. Everything seemed to pass.
### Screenshot
The sort package is still used in places that were not trivial to
change. Since Go 1.21 slices package can be uswed for sort. See
https://go.dev/doc/go1.21#slices
### Purpose
Make some progress with the migration to a more up-to-date syntax.
* main:
fix(syncthing): ensure both config and data dirs exist at startup (fixes#10126) (#10127)
fix(versioner): fix perms of created folders (fixes#9626) (#10105)
refactor: use slices.Contains to simplify code (#10121)
Jakob Borg
Catfriend1
Syncthing Release Automation
ardevd
Simon Frei
tomasz1986
yparitcherandJakob Borg
Ross Smith II
Marcel Meyer
Luke Hamburg