This improves performance when running with fsync enabled. The key
observation is that the OS may coalesce multiple concurrent fsyncs, so
we gain some performance by issuing them in parallell. Since they run
from the finisher routine, a simple fix is to run multiple finisher
routines. That's the `after` line in the graph below. The other step is
to do the same for the directory fsyncs, issuing them concurrently with
a limiter. In both cases I used the Copiers value as the concurrency
factor. Additionally, add some buffering to the channels between
routines to minimise stalls where a routine needs to wait for another.
This is the `after2` line.
All in all, this speeds up syncing 25k tiny files from 190s to 130s, a
30% improvement.
<img width="821" height="540" alt="Screenshot 2026-07-25 at 22 58 34"
src="https://github.com/user-attachments/assets/18272c89-a99e-4ee4-9b5e-b278967ea7c6"
/>
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
This tweaks the casefs cache to do in-place updates for modifications we
perform, instead of dropping the entire cache. That is, we add, remove
and rename things in the cache after having done the corresponding
operation.
We still drop the cache entirely on larger operations like MkdirAll, and
before Walk, and the 1s expiry is unchanged.
The result is performance much closer to the original. I set up a device
with 150 000 tiny files and measured the time it took for a blank device
to come online and sync all the files, with default settings apart from
`fsync` being disabled, as it otherwise dominated the benchmark...
There are three runs, all are identical in the first 40 seconds which is
the index transmission & reception, then we can see the `sensitive` run
(casefs disabled) finishes at 137 seconds (137-40 = 97s spent syncing);
the `before` run finishes at 238s (198s spent syncing); the `after` run
finishes at 142 seconds (102s spent syncing). So, effectively the sync
time is cut in half, and is now very close to having case sensitive
enabled.
<img width="812" height="476" alt="Screenshot 2026-07-25 at 23 22 22"
src="https://github.com/user-attachments/assets/0aa39712-9fd7-46cf-969e-b6a20f150a3c"
/>
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
There was a rare, tricky race condition in the setup, where a pull could
get started, find the blocks from the file we injected in
updateLocalsFromScanning, copy them into a temp file, then fail the pull
because the file was already in place. Then later, after the revert,
another pull would start, find the temp file and reuse it, making it so
needed=0 instead of the expected needed>0 the test checks for.
This avoids all that by setting the local index data prior to getting it
from the "remote" device, thus avoiding the initial pull and temp file
creation etc.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The tooltip on the dismiss button for notifications confused me a bit
(wdym mean "may" recur?) so here is an attempt at a clearer message.
Signed-off-by: tbodt <tbodt@tbodt.com>
adds a bottom margin to buttons inside panel footers, ensuring a
comfortable gap between wrapped button rows. (Fixes#10689 )
Signed-off-by: rohitanwar <mst10041967@gmail.com>
Only run rename detection for new files. This skips an expensive check
for all updates to existing files. The tradeoff is that we no longer
immediately detect renames on top of another file as a rename -- this
may instead become a copy+delete operation on the destination.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Effectively nobody uses it, it cost more than it was worth, the code
doesn't carry its weight. Remove it and let any similar mechanism in the
future be an internal implementation detail.
Closes#10812
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Adds a `lazyCollapse` directive that renders panel body content only
while expanded. Collapsed panels contribute zero watchers — content is
added to the DOM on expand (`show.bs.collapse`) and removed after the
collapse animation completes (`hidden.bs.collapse`).
Signed-off-by: Finomosec <1665799+Finomosec@users.noreply.github.com>
Previously we'd skip the health check for an up-to-date folder, so it
would continue looking up-to-date even when the folder path missing.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The recent change to disallow following symlinks by default conflicts
with the expected behavior of .stignore. This adds a new flag which may
be passed to OpenFile to skip default symlink-forbidding open flag.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
We used a size computation to generate reasonably sized batches, but the
ProtoSize call is fairly expensive as it requires a conversion to a wire
type etc. We don't need that much precision. Instead, just limit to 1000
files or 5000 blocks, which is likely approximately that much data
anyway.
Closes#10707
Signed-off-by: Jakob Borg <jakob@kastelo.net>
The counts needs to be modified manually since we're not running
triggers during migrations. The schema version needed to be bumped.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
While preparing the command, attempt to verify that the template
expansion happens in a way that will result in a non-shell-injection
command. I don't presume to say that this is a 100% prevention, and the
script itself can always do dumb shit with the file path later.
Nonetheless, we should make a best-effort attempt.
Equally, this could generate false positives for commands that are
strangely written but in fact safe. I think this is acceptable; external
versioning is currently used by approximately 0.02% of users, and
presumably most of them have a setup that is sane.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Resp. directories for the database, and both dirs and symlinks for
`FileInfo.FileSize`. Instead handle it in the UI progress percentage. We
even already have special cases there for deletions, might as well
handle directories just like any other zero-sized needed item there.
This went through the very thorough testing of running it on my laptop,
the migration was applied and it seemed to be working fine after.
---------
Signed-off-by: Simon Frei <freisim93@gmail.com>
During initial folder scan and the scan is interrupted with a
context.Canceled error, it was previously logged as a "Failed initial
scan" error, which can be misleading
The change adds a explicit check for context.Canceled and silently
ignores it (fixes#10363)
Signed-off-by: Henrik Bråthen <henrikbs@proton.me>
Verify that block requests have a hash and that it's correct. This helps
prevent certain races and ensure that only expected data is ever
returned in response to a request.
(In Syncthing prior to 1.28.1 the block hash was omitted for encrypted
requests from trusted devices. This breaks compatibility with that
specific config on those versions.)
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Ensure file was a file before the shortcut as well as after... (This was
implied when talking to a correct implementation, but not enforced.)
Make our file opening operations safe by default by ensuring the last
path component is not a symlink.
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
We had a few places where we had perhaps too much of an opinion on the
permissions on created files and directories, sometimes fuled by a
misconception about how permissions work in both Unix and Windows. Recap
on the ground rules:
- On all unixes, all file & directory creation (`Mkdir`, `MkdirAll`,
`Create`, `WriteFile`, `Open`) has the given permission bits filtered
via the user's umask. The proper permissions for us to use are in almost
all cases 0o666 for files and 0o777 for directories, strange as that may
look at the call site.
- On Windows, there is no umask but in turn all of the permission bits
except the user write bit are ignored. The absence of user write bit is
converted into the read only attribute. This means that what is proper
for Unix above is also proper for Windows.
- We make an exception when creating files for certificate keys and the
config / database directories, as those contain secrets we think should remain closed
even if the user generally collaborates with other users on the system.
(Also removal of a bugfixed copy of MkdirAll for Windows that hasn't
been necessary for a few years.)
---------
Signed-off-by: Jakob Borg <jakob@kastelo.net>
Jakob BorgandGitHub
Simon FreiandGitHub
Syncthing Release Automation
tbodtandGitHub
Rohit TanwarandGitHub
FredericandGitHub
Andrew GunnersonandGitHub
Henrik BråthenandGitHub
