Commit Graph
7526 Commits
Author SHA1 Message Date
Jakob BorgandGitHub 39df3173d4 chore(model): log sequence anomaly when update appears not to "take" (#9741)
I hope this doesn't fire, but 👻  I'm Seeing Things I Can't Explain 👻
2024-09-29 15:04:06 +00:00
maxice8andGitHub 429672e0b4 docs(docker): add healthcheck to docker-compose (#9742)
### Purpose

Syncthing had a healthcheck API for a while, and the example Dockerfile
for it has it in the form of:

HEALTHCHECK --interval=1m --timeout=10s \
CMD curl -fkLsS -m 2 127.0.0.1:8384/rest/noauth/health | grep -o
--color=never OK || exit 1

Let's add it to the docker-compose as well

### Testing

I use this docker-compose.yml file to deploy via ansible (using
community.docker.docker_compose_v2) to my machine with success, using
`wait: true` in ansible for it to use `docker compose up --wait`.

```yml
- name: Enable syncthing docker
  community.docker.docker_compose_v2:
    project_src: /srv/syncthing
    wait: true
    wait_timeout: 90
```
2024-09-29 09:53:13 -05:00
605fd6d726 fix(ignore): ensure normalization of patterns and paths match (fixes #9597) (#9717)
In ignores, normalize the input when parsing it.
When scanning, normalize earlier such that the path is already
normalized when checking ignores. This requires splitting normalization
of the string from normalization of the file, as we don't want to
attempt the latter if the file is ignored.

Closes #9598

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2024-09-28 17:16:44 +02:00
Jakob BorgandGitHub 3c476542d2 fix(ur): actually send usage report directly when enabled (#9736)
There was a bug that the unique ID was not set when reporting was
enabled, and thus the reports where rejected by the server. The unique
ID got set only on startup, so next time Syncthing restarted.

This makes sure to set the unique ID when blank.
2024-09-28 17:02:05 +02:00
Jakob BorgandGitHub 31874f3ebb chore(model): remove GUI/log warning on sequence anomaly (#9738)
I can see already in our Sentry data that there are a fair amount of
these warnings, and mostly the shape of it. Asking users to report them
will likely cause a lot of reporting effort to fairly little additional
value. We can do that when/if we have something more targeted to ask
for.
2024-09-28 16:38:07 +02:00
Jakob Borg 77942747db Merge branch 'infrastructure'
* infrastructure:
  feat(stupgrades): filter returned releases per compatibility
2024-09-26 10:25:02 +02:00
Jakob Borg fe01b396ba feat(stupgrades): filter returned releases per compatibility 2024-09-26 10:22:23 +02:00
Jakob BorgandGitHub 3583949706 refactor(upgrade): rename insecureGet which is no longer insecure (#9735) 2024-09-25 15:50:22 +00:00
Jakob BorgandGitHub 23fc22ebc5 chore: add more advanced policy configuration (#9726)
This codifies a review policy which is closer to what I always
envisioned, but which isn't expressible using the normal checks in the
GitHub GUI. It would move the commit approval check from GitHub into the
policy-bot check which is already present to enforce the
conventional-commits standard. Approvals in general would still work the
same -- it's just that the bot picks it up and toggles the status
accordingly. From a GitHub side when this is enabled we'd remove the
requires-review check from there and let the bot decide that part. We
would still require builds and tests to pass of course.

There are a couple of relexations from the current policy, details in
the code but briefly:

- Changes to translations or dependencies by a trusted person don't
require review
- Trivial changes by a trusted person, explicitly marked as such, don't
require review

This enables less bureaucracy for things like adding new translated
languages and updating dependencies, and enables the trivial-change
workflow to a larger audience than, like, me, who could always just
bypass the rules by way of being admin.
2024-09-25 17:41:56 +02:00
Jakob BorgandGitHub cba163a1fd chore: enable TLS client cache for HTTPS where appropriate (#9721)
https://forum.syncthing.net/t/infrastructure-report-discovery-stuff/22819/4
v1.28.0-rc.1
2024-09-24 08:55:04 +02:00
Jakob BorgandGitHub a8e2c8edb6 fix(connections): announce PtP links again (fixes #9730) (#9731) 2024-09-23 14:32:19 +02:00
Syncthing Release Automation 3e501d9036 gui, man, authors: Update docs, translations, and contributors 2024-09-23 03:46:01 +00:00
bt90andGitHub 9ca101756d chore(ursrv): add Nix detection (#9729)
Classify the builder `nix@nix` as [Nix](https://nixos.org/)

![369684243-172cab09-df6f-449a-a638-1f0a0c080ab3](https://github.com/user-attachments/assets/37a6e0a5-bdcb-4b31-8b36-eaaa42423382)
2024-09-22 14:03:40 +02:00
bt90andGitHub a873d12c65 chore(ursrv): extend F-Droid detection (#9728)
Our f-droid apps are currently built using `vagrant@bookworm`:

![grafik](https://github.com/user-attachments/assets/172cab09-df6f-449a-a638-1f0a0c080ab3)
2024-09-22 13:48:38 +02:00
Emil LundbergandGitHub 8ff670c564 fix(gui): get version from header when not authenticated (#9724)
### Purpose

Since #8757, the Syncthing GUI now has an unauthenticated state. One
consequence of this is that `$scope.versionBase()` is not initialized
while unauthenticated, which causes the `docsURL` function to truncate
links to just `https://docs.syncthing.net`/, discarding the section
path. This currently affects at least the "Help > Introduction" link
reachable both while logged in and not. The issue is exacerbated in
https://github.com/syncthing/syncthing/pull/9175 where we sometimes want
to show additional contextual help links from the login page to
particular sections of the docs.

I don't think it's any worse to try to preserve the section path even
without an explicit version tag, than to fall back to just the host and
lose all context the link was attempting to provide.

### Testing

- On commit b1ed2802fb (before):
  - Open the GUI, set a username and log out.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/
  - Log in.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/v1.27.10/intro/gui
- On commit 44fef31780 (after):
  - Open the GUI, set a username and log out.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/intro/gui
  - Log in.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/v1.27.10/intro/gui

### Screenshots

This is a GUI change, but affecting only URLs in the markup with no
visual changes.


### Drawbacks

If a `docsURL` call generates a versionless link to a docs page that
doesn't exist on https://docs.syncthing.net - presumably because
Syncthing is not the latest version and links to a deleted page? - then
this will lead to GitHub's generic 404 page with no link to the
Syncthing docs root. Before, any versionless link would also be a
pathless link, leading to the Syncthing docs root instead of a 404 page.
2024-09-22 09:47:02 +02:00
Jakob BorgandGitHub b1ed2802fb fix(connections): skip point-to-point interfaces when listing LANs (fixes #9719) (#9720)
Point-to-point interfaces are typically VPNs and similar which, for our
purposes, do not qualify as LANs.
2024-09-21 09:27:23 +02:00
Jakob BorgandGitHub b70cb580c8 build(deps): update all dependencies (#9723) 2024-09-21 09:25:27 +02:00
Sonu Kumar SawandGitHub 28be3ba788 chore(connections): lower log level from INFO to DEBUG for "already connected to this device" messages (fixes #9715) (#9722)
### Purpose

The primary aim of this change is to minimize log clutter in production
environments. There are many lines in the logs coming from an expected
race condition when two devices connect `already connected to this
device`. These messages do not indicate errors and can overwhelm the log
files with unnecessary noise.

By lowering the logging level, we enhance the usability of the logs,
making it easier for users and developers to identify actual issues
without being distracted

### Testing
1. Build syncthing locally
2. Start two Syncthing instances
```bash
./syncthing -no-browser -home=~/.config/syncthing1
./syncthing -no-browser -home=~/.config/syncthing2
```
3. Enable the DEBUG logs from UI for `connections` package
4. Connect the synching instances by adding remote devices from the UI
5. Observe the logs for the message `XXXX already connected to this
device`

### Screenshots


![image](https://github.com/user-attachments/assets/882ccb4c-d39d-463a-8f66-2aad97010700)

## Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.
2024-09-21 07:19:58 +00:00
Jakob BorgandGitHub d4770ddc77 chore(cmd): clean up commands (#9705)
Move infrastructure related commands to under `cmd/infra` and
development stuff to `cmd/dev`. The default build command builds the
regular user facing binaries: syncthing, stdiscosrv, and strelaysrv.
2024-09-21 09:04:22 +02:00
Simon FreiandGitHub cbe1220680 chore(fs): put the caseFS as the outermost layer again (#9716)
Reasoning in comments. The main motivation is to avoid all the case
checks when walking the filesystem.

"again" as we already tried once, but it caused a major issue ragarding
mtimefs layer. The root of this problem has been fixed in the meantime
in ac8b3342a
2024-09-18 20:31:19 +02:00
André ColombandGitHub 0b95c5fa76 fix(meta): return read error in forbidden_words_test (#9706)
When reading a file fails, the error is currently swallowed / hidden.
Probably just a typo.
2024-09-18 19:12:24 +02:00
Jakob Borg 0343bca257 Merge branch 'infrastructure'
* infrastructure:
  chore(stdiscosrv): ensure incoming addresses are sorted and unique
  chore(stdiscosrv): use zero-allocation merge in the common case
  chore(stdiscosrv): properly clean out old addresses from memory
  chore(stdiscosrv): calculate IPv6 GUA
2024-09-16 09:33:15 +02:00
Syncthing Release Automation 878016db39 gui, man, authors: Update docs, translations, and contributors 2024-09-16 03:47:01 +00:00
Simon FreiandGitHub 1f4fde9525 chore(protocol): prioritize closing a connection (#9711)
The read/write loops may keep going for a while on a closing connection
with lots of read/write activity, as it's random which select case is
chosen. And if the connection is slow or even broken, a single
read/write
can take a long time/until timeout. Add initial non-blocking selects
with only the cases relevant to closing, to prioritize those.
2024-09-15 21:13:56 +02:00
Jakob Borg 5b9d8a838f chore(stdiscosrv): ensure incoming addresses are sorted and unique 2024-09-15 17:01:16 +02:00
Jakob Borg 8b19cb1e11 chore(stdiscosrv): use zero-allocation merge in the common case 2024-09-15 15:26:40 +02:00
Jakob Borg ce1e259bb4 chore(stdiscosrv): properly clean out old addresses from memory 2024-09-15 14:20:59 +02:00
Jakob BorgandGitHub 2238a288d9 fix(model): shut down index sender faster (#9704) 2024-09-15 11:37:49 +02:00
Jakob Borg c8ee2a5cf6 chore(stdiscosrv): calculate IPv6 GUA 2024-09-15 10:48:16 +02:00
Ross Smith IIandGitHub 1704827d04 chore(gui): update HumanDuration.js (#9710)
Relevant changes:

ko: Use correct names for month and hour in Korean (465eaed)
Hide unit count if 2 in Arabic (f90d847)
2024-09-15 10:21:18 +02:00
Jakob Borg a156e88eef Merge branch 'infrastructure'
* infrastructure:
  chore(stdiscosrv): hide internal/undocumented flags
  chore(stdiscosrv): remove legacy replication
  chore(stdiscosrv): clean up s3 handling
  chore(stdiscosrv): less garbage in statistics
  chore(stdiscosrv): improve expire, logging
  chore(stdiscosrv): sched in loop
  chore(stdiscosrv): database writing logging
  chore(stdiscosrv): use order-preserving expire
  chore(stdiscosrv): simplify sorting
  chore(stdiscosrv): reduce allocations in cert handling
  chore(stdiscosrv): reduce unnecessary allocations in merge
  feat(stdiscosrv): enable HTTP profiler
  feat(discosrv): in-memory storage with S3 backing
  feat(stdiscosrv): make compression optional (and faster)
2024-09-13 08:50:24 +02:00
Jakob Borg 94d0195b63 chore(stdiscosrv): hide internal/undocumented flags 2024-09-13 08:49:13 +02:00
Jakob Borg 1616edcee3 chore(stdiscosrv): remove legacy replication 2024-09-13 08:49:13 +02:00
Jakob Borg 6505e123bb chore(stdiscosrv): clean up s3 handling 2024-09-13 08:48:04 +02:00
Jakob Borg 63e4659282 chore(stdiscosrv): less garbage in statistics 2024-09-13 08:48:04 +02:00
Jakob Borg f3f5557c8e chore(stdiscosrv): improve expire, logging 2024-09-13 08:48:04 +02:00
Jakob Borg b794726e1f chore(stdiscosrv): sched in loop 2024-09-13 08:48:04 +02:00
Jakob Borg 3d59740a0a chore(stdiscosrv): database writing logging 2024-09-13 08:48:04 +02:00
Jakob Borg 66fb65b01f chore(stdiscosrv): use order-preserving expire 2024-09-13 08:48:04 +02:00
Jakob Borg 5c2fcbfd19 chore(stdiscosrv): simplify sorting 2024-09-13 08:48:03 +02:00
Jakob Borg f9b72330a8 chore(stdiscosrv): reduce allocations in cert handling 2024-09-13 08:48:03 +02:00
Jakob Borg 822b6ac36b chore(stdiscosrv): reduce unnecessary allocations in merge 2024-09-13 08:48:03 +02:00
Jakob Borg 77f7778292 feat(stdiscosrv): enable HTTP profiler 2024-09-13 08:48:03 +02:00
Jakob Borg aed2c66e52 feat(discosrv): in-memory storage with S3 backing 2024-09-13 08:48:03 +02:00
Jakob Borg 68a1fd010f feat(stdiscosrv): make compression optional (and faster) 2024-09-13 08:33:03 +02:00
Simon FreiandGitHub ac8b3342ac chore(fs): only cache the cache for case FS, not the entire FS (#9701)
This would have addressed a recent issue that arose when re-ordering our
"filesystem layers". Specifically moving the caseFilesystem to the
outermost layer. The previous cache included the filesystem, and as such
all the layers below. This isn't desirable (to put it mildly), as you
can create different variants of filesystems with different layers for
the same path and options. Concretely this did happen with the mtime
layer, which isn't always present. A test for the mtime related breakage
was added in #9687, and I intend to redo the caseFilesystem reordering
after this.

Ref: #9677
Followup to: #9687
2024-09-12 20:35:21 +02:00
0ea90dd932 build: add generating compat.json (#9700)
This is to add the generation of `compat.json` as a release artifact. It
describes the runtime requirements of the release in question. The next
step is to have the upgrade server use this information to filter
releases provided to clients. This is per the discussion in #9656

---------

Co-authored-by: Ross Smith II <ross@smithii.com>
v1.27.13-rc.1
2024-09-11 09:29:49 +02:00
Jakob BorgandGitHub 718b1ce2b7 chore(discovery,upgrade): use regular TLS certificate verification (#9673)
This changes the two remaining instances where we use insecure HTTPS to
use standard HTTPS certificate verification.

When we introduced these things, almost a decade ago, HTTPS certificates
were expensive and annoying to get, much of the web was still HTTP, and
many devices seemed to not have up-to-date CA bundles.

Nowadays _all_ of the web is HTTPS and I'm skeptical that any device can
work well without understanding LetsEncrypt certificates in particular.

Our current discovery servers use hardcoded certificates which has
several issues:
- Not great for security if it leaks as there is no way to rotate it
- Not great for infrastructure flexibility as we can't use many load
balancer or TLS termination services
- The certificate is a very oddball ECDSA-SHA384 type certificate which
has higher CPU cost than a more regular certificate, which has real
effects on our infrastructure

Using normal TLS certificates here improves these things.

I expect there will be some very few devices out there for which this
doesn't work. For the foreseeable future they can simply change the
config to use the old URLs and parameters -- it'll be years before we
can retire those entirely.

For the upgrade client this simply seems like better hygiene. While our
releases are signed anyway, protecting the metadata exchange is _better_
and, again, I doubt many clients will fail this today.
2024-09-11 09:29:19 +02:00
Simon FreiandGitHub 29f7510f5a lib/fs: Add test reproducing missing mtimefs issue (ref #9677) (#9687)
The test is quite odd and specific, but it does reproduce the issue that
caused #9677, so I'd propose to add it to have a simple regression test
for the basic scenario. Also the option to the fakefs might come handy
for other scenarios where you want to quickly test some behaviour on a
filesystem without nanosecond precision, without actually needing access
to one.
2024-09-10 13:36:17 +02:00
Syncthing Release Automation a7f9ed4a80 gui, man, authors: Update docs, translations, and contributors 2024-09-09 03:45:15 +00:00