Commit Graph
2309 Commits
Author SHA1 Message Date
Simon FreiandGitHub 71c8a2c36f fix(db): remove invalid member from FileMetadata (#10180) 2025-06-15 09:12:25 +02:00
Simon FreiandGitHub e4ab7b4ff3 fix(watchaggregator): properly handle sub-second watch durations (fixes #9927) (#10179)
I'll let Audrius words from the ticket explain this :)

> I'm a bit lost, time.Duration is an int64, yet watcher delay is float,
> anything sub 1s gets rounded down to 0, so you just end up going into
an
> infinite loop.


https://github.com/syncthing/syncthing/issues/9927#issuecomment-2967736106
2025-06-14 00:16:22 +02:00
Simon FreiandGitHub 7b319111d3 fix: track invalid files in LocalFlags to fix global count (#10170)
Move the "invalid" bit to a local flag, making it easier to track in counts etc.
2025-06-13 07:33:31 +02:00
Simon FreiandGitHub cb7cea93a2 chore(model): remove redundant removal of internal fields in indexsender (#10173)
While it doesn't hurt, it's unnecessary since the big protobuf
modernisation, that also introduced types separate from the generated
ones for internal use. Those fields are already dropped when converting
to the wire in protocol.
2025-06-12 22:08:21 +00:00
Jakob BorgandGitHub 8afc9855f2 feat: use Ed25519 keys for sync connections (#10162)
This updates our key generation to use Ed25519 keys/certificates for
sync connections. Certificates for browser use remain ECDSA for wider
compatibility.

Ed25519 is more modern and has fewer concerns for the future than the
ECDSA curves we used previously. It is supported from Go 1.13 and
forwards, which is Syncthing 1.3.0 (October 2019).
2025-06-09 05:48:01 +00:00
Ross Smith IIandGitHub 064213ceb8 fix(fs): check for unsupported error on modern Windows (fixes #10164) (#10165)
### Purpose

Locally, on Windows 11, and on the windows-2025 GitHub runner (go 1.23
and 1.24), the `TestCopyRange` test is failing with `The request is not
supported.`

On windows-2022 and windows-2019:
```go
err == syscall.ENOTSUP
```
worked, but on Windows 11 and windows-2025, we need:
```go
errors.Is(err, errors.ErrUnsupported)
```

### Testing

Tested on Windows 11, windows-2019, windows-2022, and
[windows-2025](https://github.com/rasa/syncthing/actions/runs/15525123437/job/43703630634#step:7:2811).
2025-06-09 07:20:46 +02:00
Jakob Borg 10d20c4800 chore(fs): linter complaints 2025-06-06 13:45:44 +02:00
Jakob Borg 700bb75016 chore(model): the easier linter complaints 2025-06-06 13:45:44 +02:00
Jakob Borg 8d37e8f307 Merge branch 'main' into v2
* main:
  feat(config): expose folder and device info as metrics (fixes #9519) (#10148)
  chore: add issue types to GitHub issue templates
  build: remove schedule from PR metadata job
  chore(protocol): only allow enc. password changes on cluster config (#10145)
  chore(protocol): don't start connection routines a second time (#10146)
2025-05-31 17:10:00 +02:00
Jakob BorgandGitHub 1fdf07933c feat(config): expose folder and device info as metrics (fixes #9519) (#10148)
Tihs makes it easier to use metrics based on device and folder labels,
names, and other attributes. Other metrics which are based on folder or
device ID can be joined with these info metrics to enrich their label
sets.

```
# HELP syncthing_config_device_info Provides additional information labels on devices
# TYPE syncthing_config_device_info gauge
syncthing_config_device_info{device="I6KAH76-66SLLLB-5PFXSOA-UFJCDZC-YAOMLEK-CP2GB32-BV5RQST-3PSROAU",introducer="false",name="s1",paused="false",untrusted="false"} 1

# HELP syncthing_config_folder_info Provides additional information labels on folders
# TYPE syncthing_config_folder_info gauge
syncthing_config_folder_info{folder="default",label="The default folder",path="s2",paused="false",type="sendreceive"} 1
```

With this you can e.g. query for

```
syncthing_connections_active * on(device) group_left syncthing_config_device_info
```

Fixes #9519 
Closes #10074 
Closes #10147
2025-05-31 17:09:23 +02:00
Jakob BorgandGitHub 8e934a8c69 feat(config): enable multiple connections by default (#10151)
This changes the default number of connections from one to three (one
metadata + two data connections). This should give some advantages of
multiple connections, while also not being an overwhelming change for
larger installations. (Though those may need to tweak their settings
anyway, as always.)
2025-05-30 22:04:55 +02:00
6765867a2e chore(protocol): only allow enc. password changes on cluster config (#10145)
In practice we already always call SetPassword and ClusterConfig
together. However it's not just "sensible" to do that, it's required: If
the passwords change, the remote device needs to know about that to
check that the enc. setup is valid/consistent (e.g. tokens match,
folder-type is appropriate, ...).
And with the passwords set later, there's no point in adding them as
part of creating a new connection.

This is a "followup" (if one can call it that 4 years later :) ) to
resp. fix for the following commit:
924b96856f

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-30 09:52:47 +02:00
Simon FreiandGitHub 4fb8ee6a6f chore(protocol): don't start connection routines a second time (#10146) 2025-05-30 06:28:42 +00:00
Jakob Borg 79bac43800 Merge branch 'main' into v2
* main:
  build: properly propagate build tags to Debian build (#10144)
  fix(protocol): avoid deadlock with concurrent connection start and close (#10140)
  build: add labeler workflow for PRs (#10143)
  build(deps): update our notify package from upstream (#10142)
  build(deps): update dependencies (#10141)
  docs: general notes about v2 coming (#10135)
2025-05-29 17:10:03 +02:00
Jakob BorgandGitHub 3bd2bff23b fix(protocol): avoid deadlock with concurrent connection start and close (#10140) 2025-05-29 14:56:58 +00:00
Jakob Borg bb91f53641 Merge branch 'main' into v2
* main:
  refactor: use slices package for sorting (#10136)
  build: handle multiple general release notes
  build: no need to build on the branches that just trigger tags
2025-05-26 21:40:54 +02:00
Marcel MeyerandGitHub 598915193a refactor: use slices package for sorting (#10136)
Few more complicated usages of the sort packages are left.

### Purpose

Make progress towards replacing the sort package with slices package.
2025-05-26 20:37:49 +02:00
Jakob Borg 54f6b5c2ee Merge branch 'main' into v2
* main:
  build: use specific token for pushing release tags
  fix(gui): update `uncamel()` to handle strings like 'IDs' (fixes #10128) (#10131)
  refactor: use slices package for sort (#10132)
  build: process for automatic release tags (#10133)
  chore(gui, man, authors): update docs, translations, and contributors
2025-05-26 14:22:30 +02:00
Marcel MeyerandGitHub 48b757cac1 refactor: use slices package for sort (#10132)
The sort package is still used in places that were not trivial to
change. Since Go 1.21 slices package can be uswed for sort. See
https://go.dev/doc/go1.21#slices

### Purpose

Make some progress with the migration to a more up-to-date syntax.
2025-05-26 13:37:26 +02:00
Jakob Borg 99b707c141 fix(versioner): correct fs creation in test 2025-05-25 22:03:19 +02:00
Jakob Borg 39d6692109 Merge branch 'main' into v2
* main:
  fix(syncthing): ensure both config and data dirs exist at startup (fixes #10126) (#10127)
  fix(versioner): fix perms of created folders (fixes #9626) (#10105)
  refactor: use slices.Contains to simplify code (#10121)
2025-05-25 10:00:53 +02:00
Ashish BhateandGitHub 1a131a56f2 fix(versioner): fix perms of created folders (fixes #9626) (#10105)
As suggested in the linked issue, I've updated the versioner code to use
the permissions of the corresponding directory in the synced folder,
when creating the folder in the versions directory

### Testing
- Some tests are included with the PR. Happy to add more if you think
there are some edge-cases that we're missing.
- I've tested manually on linux to confirm the permissions of the
created directories.
- I haven't tested on Windows or OSX (I don't have access to these OS)
2025-05-24 07:35:32 +02:00
pullmergeandGitHub beda37f28b refactor: use slices.Contains to simplify code (#10121)
There is a [new function](https://pkg.go.dev/slices@go1.21.0#Contains)
added in the go1.21 standard library, which can make the code more
concise and easy to read.
2025-05-23 10:36:06 +00:00
Jakob Borg 78bfe643a8 chore: bump config version, prevent accidental downgrade 2025-05-20 15:37:19 +02:00
Jakob Borg 0a58747eb2 chore: further minor lint fixes 2025-05-20 15:04:33 +02:00
Jakob Borg 96b03fac04 chore: trivial lint fix 2025-05-20 14:34:20 +02:00
Jakob BorgandGitHub 085455d72e feat: add syncthing debug database-statistics command (#10117)
This adds a command that shows database statistics. Currently it
requires a fork of the sqlite package to add the dbstats virtual table;
the modernc variant already has it.

This also provides the canonical mapping between folder ID and database
file, for tinkerers...

```
% ./bin/syncthing debug database-statistics
DATABASE                 FOLDER ID    TABLE                                  SIZE     FILL
========                 ====== ==    =====                                  ====     ====
main.db                  -            folders                               4 KiB    8.4 %
main.db                  -            folders_database_name                 4 KiB    6.0 %
main.db                  -            kv                                    4 KiB   41.1 %
main.db                  -            schemamigrations                      4 KiB    3.9 %
main.db                  -            sqlite_autoindex_folders_1            4 KiB    3.7 %
...
folder.0007-txpxsvyd.db  w3ejt-fn4dm  indexids                              4 KiB    1.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  kv                                    4 KiB    0.8 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  mtimes                              608 KiB   81.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  schemamigrations                      4 KiB    3.9 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_blocklists_1      4108 KiB   89.5 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_blocks_1        700020 KiB   88.1 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_devices_1            4 KiB    3.6 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_autoindex_kv_1                 4 KiB    0.6 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_schema                        12 KiB   45.9 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_sequence                       4 KiB    1.0 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_stat1                          4 KiB   12.2 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  sqlite_stat4                          4 KiB    0.2 %
folder.0007-txpxsvyd.db  w3ejt-fn4dm  (total)                         1906020 KiB   92.8 %
main.db + children       -            (total)                         2205888 KiB   92.0 %
```
2025-05-20 14:27:08 +02:00
Jakob Borg 72849690c9 fix(model): index handler error handling 2025-05-20 14:26:14 +02:00
Jakob BorgandGitHub 964c8d7d65 fix(model): correct bufferpool handling; simplify (#10113)
The copier routine refactor resulted in bad buffer pool handling,
putting a buffer back into the pool twice. This simplifies and removes
the danger prone Upgrade() method.
2025-05-16 22:50:13 +02:00
Jakob Borg bacf506e90 Merge branch 'main' into v2
* main:
  chore(gui, man, authors): update docs, translations, and contributors
  fix(config): mark audit log options as needing restart (fixes #10099) (#10100)
  fix(config): deep copy configuration defaults (fixes #9916) (#10101)
2025-05-16 16:17:43 +02:00
8c3d2f3bc5 fix(config): mark audit log options as needing restart (fixes #10099) (#10100)
### Testing

Change the `auditEnabled` option and you should get a prompt in the Web
GUI.
Restart and change the `auditFile` option, and you should get that same
prompt.

The prompt you should get is shown in the screenshots below.

### Screenshots


![Screenshot_20250507_122546](https://github.com/user-attachments/assets/23ce7c42-5e60-4f88-ac58-f312a9a1f5cc)

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-09 10:49:11 +00:00
Hazem KrimiandGitHub 702ed8ecc1 fix(config): deep copy configuration defaults (fixes #9916) (#10101)
### Purpose

Setting default configuration was not working properly since the
defaults struct is not deeply copied.

### Testing

Try running commands to change default configuration and either inspect
`config.xml` or `/rest/config` result to see the applied changed.
Example:
```
./syncthing cli config defaults folder versioning params set keep 5
```
2025-05-09 07:40:32 +02:00
Simon FreiandGitHub 821d6f43ac chore(model): refactor copier for more flatness (#10094)
Flattened the copier code more. Also removing and moving some
parameters/return values to simplify things. Generally rely less on
return values, e.g. by handling errors right away and using `state` to
do the right thing (e.g. abort on failure).

Supposed to be a refactor without any behaviour changes, except for
fixing a tiny regression on folder order: We used to try copying from
the same folder first, but lost that property at some point (also sent a
PR fixing only that, I'd merge that first making this refactor only).
2025-05-04 09:23:57 +02:00
Simon FreiandGitHub fa7b81e1cf fix(model): use same folder first in copier (#10093)
Where `folderFilesystems` and `folders` is built, there's a comment
spelling out the purpose: To have the same folder first, as that's the
most likely to get hits. Plus a copy is possibly more efficient than
from another folder, e.g. if that's on a different filesystem. We lost
that behaviour during some unrelated change.

(Also sneaking in a comment fix on yesterdays change.)
2025-05-02 13:15:26 +02:00
Simon FreiandGitHub 6b94599467 chore(db, model): simplify per hash DB lookup in copier (#10080)
This is a draft because I haven't adjusted all the tests yet, I'd like
to get feedback on the change overall first, before spending time on
that.

In my opinion the main win of this change is in it's lower complexity
resp. fewer moving parts. It should also be faster as it only does one
query instead of two, but I have no idea if that's practically
relevant.

This also mirrors the v1 DB, where a block map key had the name
appended. Not that this is an argument for the change, it was mostly
reassuring me that I might not be missing something key here
conceptually (I might still be of course, please tell me :) ).

And the change isn't mainly intrinsically motivated, instead it came
up while fixing a bug in the copier. And the nested nature of that code
makes the fix harder, and "un-nesting" it required me to understand
what's happening. This change fell out of that.
2025-05-01 13:44:25 -05:00
f183d1cbec chore(syncthing): ensure migrated database is closed before exiting (#10076)
After opening the database, we performed some checks, such as whether
the migration had already been successfully completed. If so, the
function returned immediately, and the database was not closed.

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-05-01 18:36:35 +00:00
Simon FreiandGitHub 58bf2b5515 fix(model): close fd immediately in copier (#10079) 2025-05-01 10:15:02 -05:00
Jakob Borg abe34fc1f6 Merge branch 'main' into v2
* main:
  feat(config): add option for audit file (fixes #9481) (#10066)
  chore(api): log X-Forwarded-For (#10035)
  chore(gui): update dependency copyrights, add script for periodic maintenance (#10067)
  chore(gui, man, authors): update docs, translations, and contributors
  chore(syncthing): remove support for TLS 1.2 sync connections (#10064)
  fix(osutil): give threads same I/O priority on Linux (#10063)
  chore(stun): switch lookup warning to debug level
  chore(gui, man, authors): update docs, translations, and contributors
2025-04-24 08:47:52 +07:00
Simon FreiandGitHub be002362b3 fix(model): loop-break regression while block copying in puller (#10069) 2025-04-24 08:29:30 +07:00
Marcus B SpencerandGitHub 190dff142c feat(config): add option for audit file (fixes #9481) (#10066) 2025-04-23 22:32:23 +07:00
c667ada63a chore(api): log X-Forwarded-For (#10035)
### Purpose

Fix https://github.com/syncthing/syncthing/issues/9336

The `emitLoginAttempt` function now checks for the presence of an
`X-Forwarded-For` header. The IP from this header is only used if the
connecting host is either on loopback or on the same LAN.

In the case of a host pretending to be a proxy, we'd still have both IPs
in the logs, which should make this much less critical from a security
standpoint.

### Testing

1. directly via localhost
2. via proxy an localhost

#### Logs

```
[3JPXJ] 2025/04/11 15:00:40 INFO: Wrong credentials supplied during API authorization from 127.0.0.1
[3JPXJ] 2025/04/11 15:03:04 INFO: Wrong credentials supplied during API authorization from 192.168.178.5 proxied by 127.0.0.1
```

#### Event API

```
  {
    "id": 23,
    "globalID": 23,
    "time": "2025-04-11T15:00:40.578577402+02:00",
    "type": "LoginAttempt",
    "data": {
      "remoteAddress": "127.0.0.1",
      "success": false,
      "username": "sdfsd"
    }
  },
  {
    "id": 24,
    "globalID": 24,
    "time": "2025-04-11T15:03:04.423403976+02:00",
    "type": "LoginAttempt",
    "data": {
      "proxy": "127.0.0.1",
      "remoteAddress": "192.168.178.5",
      "success": false,
      "username": "sdfsd"
    }
  }
```

### Documentation

https://github.com/syncthing/docs/pull/907

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2025-04-23 06:01:13 +00:00
Jakob BorgandGitHub 50480b89fc chore(syncthing): remove "default" folder concept (#10068)
This removes the creation of the `default` folder on startup. My feeling
is that the concept is not widely used or appreciated.
2025-04-23 05:59:24 +00:00
Jakob BorgandGitHub ff33d976d1 chore(syncthing): remove support for TLS 1.2 sync connections (#10064)
This cleans up the option to allow old TLS 1.2 sync connections. The
flag existed for compatibility with old Syncthing versions that don't
support TLS 1.3, which is approximately Syncthing 1.2.2 (September 2019)
and older. ("Approximately" because it depends on the Go version it's
built with and that's when we switched to building with Go 1.13.)

Ref #10062 because it reminded me this exists.
2025-04-21 10:30:43 +07:00
TheCreeperandGitHub 69890b4282 fix(osutil): give threads same I/O priority on Linux (#10063) 2025-04-21 02:30:52 +00:00
Jakob Borg 533c9a6ab0 chore(stun): switch lookup warning to debug level 2025-04-17 07:29:10 +07:00
Jakob Borg ed6575411f Merge branch 'main' into v2
* main:
  feat(stdiscosrv): configurable desired not-found rate
  chore(blobs): generalised blob storage
  chore(stdiscosrv): path style s3
  feat(ursv): add os/arch/distribution metric
  chore(strelaypoolsrv): limit number of returned relays
  build(infra): run in Docker environment for pushes
  chore(stupgrades): expose latest release as a metric
  feat(api, gui): allow authentication bypass for metrics (#10045)
2025-04-13 09:44:09 +02:00
Jakob Borg b5ffd0a796 chore(blobs): generalised blob storage 2025-04-13 09:41:16 +02:00
Jakob Borg c74299b59a chore(stdiscosrv): path style s3 2025-04-13 09:40:14 +02:00
Jakob Borg 8b6d837483 feat(ursv): add os/arch/distribution metric 2025-04-13 09:40:14 +02:00
Sébastien WENSKEandGitHub ab5c42f4a0 feat(api, gui): allow authentication bypass for metrics (#10045)
### Purpose

Give the ability to skip authentication for prometheus metrics
("/metrics").

### Testing

When authentication is enabled and "Metrics Without Auth" is checked
(not the default), the "/metrics" path remains accessible even when
disconnected.

### Screenshots


![image](https://github.com/user-attachments/assets/144b696b-dd72-46f4-94d5-cd21848e4a4c)

### Documentation

https://github.com/syncthing/docs/pull/906
2025-04-13 07:35:57 +00:00