It's only polite, and it enables better troubleshooting in situations
like weird-ass requests to the upgrades download service.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
* infrastructure:
fix(stdiscosrv): slightly increase reannounce times and smoothing
fix(stdiscosrv): 500 errors should be accompanied by log entry
fix(ursrv): data problems are the clients fault
build: sign docker images
This is primarily to improve the experience in environments that start
Syncthing from a graphical environment, e.g., Windows.
When already running, instead of printing an error we open the GUI. If
the GUI is not available (but the lockfile indicates we are running),
print an error and exit.
If --no-browser is given or STNOBROWSER is set, act like before.
Closes#10736. Marking as `fix` because this does not deserve to trigger
a minor release on its own.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
When adding a folder in receive-only mode where the contents were
already identical to a remote device, directories would remain as
locally changed when everything else had consolidated. The reason this
happened is that we only did the matching between locally changed files
and their global equivalent for changed items, but directories are
typically not "changed" much as we don't track their mtime, so they
wouldn't pass through this stage when scanning.
Now, instead, do the check when we're anyway walking all the files in
phase two of scanning. This catches all cases of identical items, files
or directories, regardless of how they came to be in the index.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
There is a danger of deadlock wherever we have database operations
inside a database iterator. The iterator itself pins a connection, so
the nested operations need another connection; if we've reached
maxOpenConns then it blocks until a connection is released. If all
connections are consumed by such iterators then none can make progress.
Luckily, we have limited concurrency for most such iterator loops. They
are part of scanning, pulling, reverting, etc where there is only ever
one such routine per folder. The exception is block reuse in the copier
routine, which is limited by the `Copiers` setting per folder. This
could in practice deadlock since you could set copiers to eight and end
up with six `AllLocalBlocksWithHash` iterators when `maxDBConns=6`, all
of which need to make additional database calls inside the loop.
This PR fixes the problem twice;
- The problematic loop does not need to be reentrant. The set of blocks
that may be returned by the iterator is finite so we can easily just
collect them to a slice before we start processing them. This avoids the
problem entirely.
- We do not need to limit database connections as strictly as we
currently do. Increase the maximum allowed, while reducing the number of
held-open idle connections slightly. This is not an exact science, but
ideally we want "most" operations to be able to use the pinned
connections to avoid cache churn. Most operations are short lived
queries, or single-goroutine iterators with short lived queries inside,
so four connections seems like it should usually be enough. 🤷
- Additionally, set a cap on Copiers. Currently you could set it to an
arbitrarily large number, which is not advantageous. Limit it to
2*NumCPU which scales somewhat with system performance.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
This improves performance when running with fsync enabled. The key
observation is that the OS may coalesce multiple concurrent fsyncs, so
we gain some performance by issuing them in parallell. Since they run
from the finisher routine, a simple fix is to run multiple finisher
routines. That's the `after` line in the graph below. The other step is
to do the same for the directory fsyncs, issuing them concurrently with
a limiter. In both cases I used the Copiers value as the concurrency
factor. Additionally, add some buffering to the channels between
routines to minimise stalls where a routine needs to wait for another.
This is the `after2` line.
All in all, this speeds up syncing 25k tiny files from 190s to 130s, a
30% improvement.
<img width="821" height="540" alt="Screenshot 2026-07-25 at 22 58 34"
src="https://github.com/user-attachments/assets/18272c89-a99e-4ee4-9b5e-b278967ea7c6"
/>
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
This tweaks the casefs cache to do in-place updates for modifications we
perform, instead of dropping the entire cache. That is, we add, remove
and rename things in the cache after having done the corresponding
operation.
We still drop the cache entirely on larger operations like MkdirAll, and
before Walk, and the 1s expiry is unchanged.
The result is performance much closer to the original. I set up a device
with 150 000 tiny files and measured the time it took for a blank device
to come online and sync all the files, with default settings apart from
`fsync` being disabled, as it otherwise dominated the benchmark...
There are three runs, all are identical in the first 40 seconds which is
the index transmission & reception, then we can see the `sensitive` run
(casefs disabled) finishes at 137 seconds (137-40 = 97s spent syncing);
the `before` run finishes at 238s (198s spent syncing); the `after` run
finishes at 142 seconds (102s spent syncing). So, effectively the sync
time is cut in half, and is now very close to having case sensitive
enabled.
<img width="812" height="476" alt="Screenshot 2026-07-25 at 23 22 22"
src="https://github.com/user-attachments/assets/0aa39712-9fd7-46cf-969e-b6a20f150a3c"
/>
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
There was a rare, tricky race condition in the setup, where a pull could
get started, find the blocks from the file we injected in
updateLocalsFromScanning, copy them into a temp file, then fail the pull
because the file was already in place. Then later, after the revert,
another pull would start, find the temp file and reuse it, making it so
needed=0 instead of the expected needed>0 the test checks for.
This avoids all that by setting the local index data prior to getting it
from the "remote" device, thus avoiding the initial pull and temp file
creation etc.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Only run rename detection for new files. This skips an expensive check
for all updates to existing files. The tradeoff is that we no longer
immediately detect renames on top of another file as a rename -- this
may instead become a copy+delete operation on the destination.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Effectively nobody uses it, it cost more than it was worth, the code
doesn't carry its weight. Remove it and let any similar mechanism in the
future be an internal implementation detail.
Closes#10812
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Previously we'd skip the health check for an up-to-date folder, so it
would continue looking up-to-date even when the folder path missing.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The recent change to disallow following symlinks by default conflicts
with the expected behavior of .stignore. This adds a new flag which may
be passed to OpenFile to skip default symlink-forbidding open flag.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
We used a size computation to generate reasonably sized batches, but the
ProtoSize call is fairly expensive as it requires a conversion to a wire
type etc. We don't need that much precision. Instead, just limit to 1000
files or 5000 blocks, which is likely approximately that much data
anyway.
Closes#10707
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The counts needs to be modified manually since we're not running
triggers during migrations. The schema version needed to be bumped.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
While preparing the command, attempt to verify that the template
expansion happens in a way that will result in a non-shell-injection
command. I don't presume to say that this is a 100% prevention, and the
script itself can always do dumb shit with the file path later.
Nonetheless, we should make a best-effort attempt.
Equally, this could generate false positives for commands that are
strangely written but in fact safe. I think this is acceptable; external
versioning is currently used by approximately 0.02% of users, and
presumably most of them have a setup that is sane.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Verify that block requests have a hash and that it's correct. This helps
prevent certain races and ensure that only expected data is ever
returned in response to a request.
(In Syncthing prior to 1.28.1 the block hash was omitted for encrypted
requests from trusted devices. This breaks compatibility with that
specific config on those versions.)
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Ensure file was a file before the shortcut as well as after... (This was
implied when talking to a correct implementation, but not enforced.)
Make our file opening operations safe by default by ensuring the last
path component is not a symlink.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
We had a few places where we had perhaps too much of an opinion on the
permissions on created files and directories, sometimes fuled by a
misconception about how permissions work in both Unix and Windows. Recap
on the ground rules:
- On all unixes, all file & directory creation (`Mkdir`, `MkdirAll`,
`Create`, `WriteFile`, `Open`) has the given permission bits filtered
via the user's umask. The proper permissions for us to use are in almost
all cases 0o666 for files and 0o777 for directories, strange as that may
look at the call site.
- On Windows, there is no umask but in turn all of the permission bits
except the user write bit are ignored. The absence of user write bit is
converted into the read only attribute. This means that what is proper
for Unix above is also proper for Windows.
- We make an exception when creating files for certificate keys and the
config / database directories, as those contain secrets we think should remain closed
even if the user generally collaborates with other users on the system.
(Also removal of a bugfixed copy of MkdirAll for Windows that hasn't
been necessary for a few years.)
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
- Allow zero-sized requests since they are sent by all current versions
of Syncthing.
- Stop sending zero-sized requests since that's stupid.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
lib/ur brings in a lot of dependencies we don't need in e.g.
stcrashreceiver, who only needs the small failure reporting structs.
Make those part of the lean `contract` package instead.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The runtime prints a lot of context for crashes due to bad pointers etc,
which is required to understand the crash, but this context comes before
the `fatal error: ...` line. Currently those lines get filtered out and
not included in the crash report. This change modifies the criteria so
that we start collecting crash data also at a line that begins with
`runtime:`, and tweaks the parsing later to look for the specific
`panic:` or `fatal error:` which may come later as the subject.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
This makes sure the user running Syncthing, and hence Synchting itself,
has read/write/execute on directories in .stversions. The other
permission bits remain copied from the source directory, ensuring
whatever group and other permissions were set remain in effect.
Closes#10695.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The locking logic for upgrades got inverted in the lockfile changes. If
we got the lock it means Syncthing wasn't already running, so we can do
a direct upgrade. If we failed to get the lock it means Syncthing was
running and we should tell the REST interface to do the upgrade.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
* infrastructure:
fix(stcrashreceiver): allow extra pre/post data in version line
chore(stcrashreceiver): improve logging
chore(stdiscosrv): prewarm counters at startup
In the common case (sparse files enabled, not reusing old data) we'd
optimise away pulling & writing zero blocks. However in the corner cases
we'd go through the whole processing of pulling the block over the
network which is of course entirely unnecessary.
Now, instead, always take an optimised path for all-zeroes blocks. In
the clean case we do nothing, otherwise we materialise a block of zeroes
and write it directly.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
These headers should not be inspected when running with a TLS listener.
Additionally, we should really enable them individually instead of
trusting the proxy to filter out the unused variants, but baby steps.
Reported by multiple AI vuln scanners.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
This adds a new folder-level configuration `FullBlockIndex`. It controls
whether we maintain the block index for a given folder -- currently
that's always true, now it becomes possible to turn off. The block index
is used for lookup of blocks across files and folders. Effectively, when
syncing a change, for each block, we check:
1. Is the block already present in the old version of the file? If so,
we can reuse (copy) it without network transfer. **This check is always
possible.**
2. Is the block already present in any other file in this folder or
other folders? If so we can copy it. **This check is only possible with
the full block index.**
3. We must transfer the block over the network.
Maintaining the full block index is costly in time, I/O and database
size. With this PR, maintaining the full block index becomes the default
for send-receive and receive-only folders only, with it disabled for
send-only and receive-encrypted folders. The block index is never useful
for encrypted folders, as blocks are encrypted separate for each file.
It is also not useful for send-only folders by themselves, though the
data in the send-only folder could be reused by other receive-type
folders if it were enabled.
For very large folders it may make sense to disable the full block index
regardless of folder type and just accept the resulting decrease in data
reuse.
Disabling or enabling the option in the GUI causes the index to be
destroyed or rebuilt accordingly.
https://github.com/syncthing/docs/pull/1005
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Hopefully deflakes TestHTTPLogin on Windows, where it currently often
times out, presumably in the config saving stage after already having
started a shutdown of the API and being CPU constrained due to password
hashing.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The test expected the stopped scanner to produce at most numHashers
additional results, but there's also the case where a directory is
encountered (which doesn't require hashing) and sent directly.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
There was a race condition where using IndexUpdate would trigger a pull,
which would sync the delete we are looking for, making the completion
100%. By doing the insert directly into the database we are not
triggering these things and get the expected completion percentage
always.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
These have been flaky for a long time, seemingly because the multiple
connection code slightly changed the timing of cluster config sending by
moving them to the connection promotion loop. This adds some resiliency
to that, instead of assuming that the CC:s will be immediately available
after adding the connection.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
* infrastructure:
chore(stdiscosrv): smooth retry-after delays over a slightly larger normal distribution
chore(stdiscosrv): optionally delay shutdown
chore(stdiscosrv): adjust desired seen and unseen rate separately
Jakob Borg