Repository navigation
[NOTICE] Version 3.30.1 Causing Sync Issues / Synchronising All Files #13872
Description
Activity
- pinned this issue
on Oct 24, 2024 - locked and limited conversation to collaborators
on Oct 24, 2024 - unlocked this conversation
on Oct 24, 2024 The github versions don't install on devices that install from F-Droid / Gplay / ?.
Older versions 3.29.1 available as APK on F-Droid does NOT install citing invalid package ....
So without uninstall no remedy.? Gplay doesn't offer a downgrade AFAICT@noci2012, F-Droid builds are signed by F-Droid. Builds published on GitHub are signed using our key. This means that one must, unfortunately, choose and stay with one source for continued updates.
In some cases, the versions differ in some aspects due to some other constraints, e.g. Google Libraries. This is the case for Google Play and F-Droid builds.
Downgrading is not officially supported, as far as I'm aware. To install an older version, one should first remove the newer one.
Technical Background
I'm curious why the same files were repeatedly synchronized ad infinitum? I don't think the Technical Background currently addresses this.
In my case, Nextcloud downloaded 15 GB of data from a server containing 1 GB of data. Others have reported the same.
Reacted by Timon Zielonka, Vincent Rubiolo, Dudley and bobemoeI see in the F-Droid build log:
com.nextcloud.client:30300290 2024-10-22 15:20:18,028 INFO: Building version 3.30.2 (30300290) of com.nextcloud.client .
...
BUILD SUCCESSFUL in 13m 56s 56 actionable tasks: 56 executed 2024-10-22 15:36:53,598It seems that it takes 3 to 4 days until the build is distributed. Maybe we should help F-Droid to speed up the distribution process. I some knows more, please help out.
I'm curious why the same files were repeatedly synchronized ad infinitum?
I don't think so, because I have version 3.30.1 at the moment on LineageOS and it sync all files once (using almost all the battery), but at the moment it is fine. I checked it on another phone and it seems, that it sync the same file several times, but not ad infinitum. (Just my personal experience with 3 clients, so nothing to really relay on)
Possible that the repeating sync start is caused by a timeout. With a lower server space usage, the sync seems to be repeated less often.
Mitigation (i.e. reinstall, but explained really verbosely) is for technical users only.
Many users don't know their password or what a password is (an example from my circles where they just want to see pictures by swiping). For them re-logging into the app via a computer with a QR code through an obscure web UI path is impossible.
You littered every single user's phone with gigabytes of difficult-to-remove data. This is a major mess. Especially, that is comes just months after another major synchronisation problem.
I already received complaints that Nextcloud duplicates all files into its cache and filling up the phone eventually (details in #13400 (comment)).
Please, create a proper, automated solution for this problem ASAP. Otherwise you offload solving this chaos to every single technical user who chose and distributed your application.
Reacted by Felix Stupp, Xavier Bestel and thomaslpYou littered every single user's phone with gigabytes of difficult-to-remove data.
Data can be simply removed via "on device" and then removing the folder that you do not want, choosing "local only".
Please, create a proper, automated solution for this problem ASAP.
I am all for it, but do you have an idea how?
Internally the synced folders are stored in a database and we cannot differentiate between wanted and wrongly added folders there.
This is why the new overview allows user to also cancel 2way sync, but only manually.Reacted by RikkiBCReacted by Xavier BestelIn the short term an "Empty local storage" button would be a good interim solution. I can communicate to press this button and this problem goes away. This would also conveniently fix #9128, too.
As a next step I could imagine a Trashbin-like system which, after a configurable amount of used local storage (e.g. 2% of device storage), it'd warn the user to press this button.
One more step further, the application could be re-architected in a way that it outsources the internal storage handling into a regular Linux directory which the user can manage themselves. This would fix #13400. Even if you agree with this direction, I see this would have many edge cases and is not a quick solution.
68 remaining items
When I put files into a synced folder via filemanager, they are not synced. NC does not know that files.
Reacted by maxnatt@flixmart Yes. TwoWaySyncWorker is only working via Wi-Fi for now.
Are you referring to 3.30.5 explicitly or also to earlier revisions? Because with 3.30.4, it used 15 GB of my wife's mobile plan before I deinstalled it after all other countermeasures from this thread had failed.
Since version 3.30.2, the Two-Way Sync Worker only uses Wi-Fi. Could the 15GB usage be due to Auto-Upload? Was Auto-Upload enabled with mobile data for some folders?
Additionally, we fixed the re-download issue in v3.30.5. However, if only mobile data is active, the Two-Way Sync Worker does not download anything.
Was Auto-Upload enabled with mobile data for some folders?
[...]
However, if only mobile data is active, the Two-Way Sync Worker does not download anything.
Auto upload was active for the photo folder, but only allowed for wifi (and this worked for years now), plus she did not add 15 GB+ of new photos that day. And, I was able to reproduce with 3.30.4 that if I switch off wifi, the unstoppable syncing process kept running. It even re-spawned several times after deleting the app cache and other tries to stop it.
Unfortunately, I had to deinstall and reinstall the app to prevent more damage to my wife's yearly mobile plan¹, therefore I can no longer test this behavior.
¹It would anyway never have succeeded, even if I had kept the phone connected via wifi for several days: The entire nc folder was added to the sync list, way more than 100 GB of data (I wrote 100 GB before, but this was outdated information 😉), which is more than the free space on the device anyway.
Was Auto-Upload enabled with mobile data for some folders?
[...]
However, if only mobile data is active, the Two-Way Sync Worker does not download anything.
Auto upload was active for the photo folder, but only allowed for wifi (and this worked for years now), plus she did not add 15 GB+ of new photos that day. And, I was able to reproduce with 3.30.4 that if I switch off wifi, the unstoppable syncing process kept running. It even re-spawned several times after deleting the app cache and other tries to stop it.
Unfortunately, I had to deinstall and reinstall the app to prevent more damage to my wife's yearly mobile plan¹, therefore I can no longer test this behavior.
¹It would anyway never have succeeded, even if I had kept the phone connected via wifi for several days: The entire nc folder was added to the sync list, way more than 100 GB of data (I wrote 100 GB before, but this was outdated information 😉), which is more than the free space on the device anyway.
Thanks for the information. We were aware of the unstoppable syncing issue and addressed it by stopping the scheduled periodic worker at app launch in v3.30.5. We expect this issue to be resolved in v3.30.5.
As for the mobile data usage still occurring in v3.30.4, we will look into it.
Reacted by spaceChRiSI was hit by this too right at the beginning and uninstalled after my monthly allowance was consumed! I am watching this thread closely and waiting for it to become stable enough for me to reinstall.
One thing I have read is that sync is now limited to WiFi. I have no WiFi and do all my sync on Ethernet (or mobile). I think this needs to be amended to allow sync on Ethernet too (and a toggle for mobile) or it wont even work for me when I do reinstall? #13847 (review)
Reacted by svenb1234 and NovemberheksOne thing I have read is that sync is now limited to WiFi. I have no WiFi and do all my sync on Ethernet (or mobile). I think this needs to be amended to allow sync on Ethernet too (and a toggle for mobile) or it wont even work for me when I do reinstall? #13847 (review)
Adding an option to also run it on mobile data, is possible.
Ok,, with 3.30.5 you managed to break the auto-upload alltogether.
Now the status is that some folders get auto-uploaded and others dont. Namely the first folder (Camera) does not get uploaded any more. I tried everything besides de-installing and re-installing. It does not help. Camera folder does not get uploaded any more, whereas pictures from Conversations (XMPP messenger) do, but screenshots dont. Auto-upload is now really completely broken. Please everybody, check_all_ your auto-upload folders if they really still work.Ok,, with 3.30.5 you managed to break the auto-upload alltogether. Now the status is that some folders get auto-uploaded and others dont. Namely the first folder (Camera) does not get uploaded any more. I tried everything besides de-installing and re-installing. It does not help. Camera folder does not get uploaded any more, whereas pictures from Conversations (XMPP messenger) do, but screenshots dont. Auto-upload is now really completely broken. Please everybody, check_all_ your auto-upload folders if they really still work.
I don't see this behavior, but I only have custom folders set up for auto-upload. Perhaps this is related only to automatically detected folders?
Ok,, with 3.30.5 you managed to break the auto-upload alltogether. Now the status is that some folders get auto-uploaded and others dont. Namely the first folder (Camera) does not get uploaded any more. I tried everything besides de-installing and re-installing. It does not help. Camera folder does not get uploaded any more, whereas pictures from Conversations (XMPP messenger) do, but screenshots dont. Auto-upload is now really completely broken. Please everybody, check_all_ your auto-upload folders if they really still work.
I'm not seeing this myself either, but it sounds like a new issue that would be best in its own bug report if you're experiencing it.
Reacted by TomWe did not changed any thing auto-uploaded related.
Is Camera or any other in 2-way sync list?
Else I think it is indeed unrelated.2-way sync is turned off with the corresponding button. I have problems in understanding where a 2-way sync list is to be found? My understanding so far was that turning on 2-way sync affects the "standard" sync list - as it did this when the option was broken...
I really do not want to turn it on ever again, so I fail to see if there are any further config options coming up after that.@joe-average-user which version do you use?
You should be able to see it in settings -> internal two-way sync.There you can also disable it.
The list will be shown beneath the "interval".
You missed the point. your 2-way-sync implementation broke the sync as a whole. In fact every folder seems to get resynced continously, no matter what direction. I would say you misinterpret the sync-database in some point(field).
This does not happen on a fresh install, only in update path.Oh, and forgot another important thing: the sync then ignores the setup when it only should sync with WLAN connection, not mobile. This is reported several times here and I can confirm this problem.
Same here. And it cost me a month's worth of mobile data and completely filled up my space on the server. I'm not going to use it on my mobile devices anymore until I can be absolutely sure these issues are solved.
- unpinned this issue
on Jan 14, 2025
Nextcloud Android Client Version 3.30.1
Dear Community,
It is with great regret that we have to inform you that a recent version of the Nextcloud Android Client, specifically version 3.30.1, has introduced a bug that may cause one or more of the following issues:
Please note that manual intervention will be required by users who have upgraded to the affected version.
Updating - How To Disable Two-Way Synchronisation
We have carefully investigated this issue and released new versions which aim to address these issues.
With Version 3.30.4 we have introduced a new way to manage two-way sync. To access this redesigned menu open Settings → Internal two way sync.
This menu allows users to globally enable or disable two-way synchronisation. Disabling this will ensure that no two-way synchronisation will occur in the future. As before, users have the option to selectively remove individual folders using the × icon next to the folder name. For advanced users, there is also an option to set the interval at which the Nextcloud Android Client will attempt two-way synchronisation.
We have also added a button to disable two-way synchronisation for all folders. This option will appear in the overflow menu (⋮) if two-way synchronisation is enabled for at least one folder. Furthermore, this will attempt to cancel any transfers in progress and update the database to exclude those folders from being synchronised again.
Removing Local Files
Some users have experienced increased storage usage on their device. Allocated storage space can be freed by removing local files: open the On device tab and long press on any file or folder. Then open the overflow menu (⋮) to Select all files. When all files are selected, re-open the overflow menu and select Delete. When prompted, select Local only. This is important, as selecting Delete in this dialogue will also remove all selected files from the server. The removal process will take some time depending on how many files have been downloaded.
How to Upgrade
The Nextcloud Android Client can be updated via your preferred installation method. All versions are also available on GitHub for download.
As of 05/11/2024, version 3.30.4 is available from F-Droid. This version includes all major fixes and the redesigned management interface.
Unfortunately, the Google Play release will take some more time to be available as we are currently blocked by Google. This is, however, tracked in a different issue: #13871.
We will update this issue as soon as the releases are available.
Mitigation
We strongly recommend that all users to upgrade to version 3.30.4 or later using your preferred method to resolve these issues. This version includes several fixes and changes that should resolve any issues related to two-way synchronisation. Please refer to the explanation above for information. If this version is not available for you, or if something else is blocking this update, there are several options to mitigate any unwanted behaviour.
Option 1 - Manual Removal
From version 3.30.2, users have the option to quickly check and stop two-way synchronisation for specific folders. To access this menu open Settings → Internal two way sync. On the following screen one will see all folders scheduled for two-way synchronisation and have the option to exclude folders using the × icon next to the folder name. If a blank page is displayed instead, no folders are scheduled for two-way synchronisation.
Option 2 - Reinstalling
For version 3.30.1 to 3.30.3 (outdated)
The simplest option is to reinstall the Nextcloud Android Client. Due to the technical nature of the issue reinstalling any version above 3.30.0 will resolve the issue. The process of reinstalling an app varies from Android version to Android version. However, the general process will be similar to the following:
Please note that any local data of the Nextcloud Android Client that is present on the mobile device will be lost during the removal process.
Option 3 - Reinitialise Local Account
For version 3.30.1 to 3.30.3 (outdated)
Another option, if you prefer to keep the current version installed, is to remove the current user account and it add again. This should have a similar effect to reinstalling the application. To re-initialise your account in the Nextcloud Android Client, follow these steps:
Please note that any local data of the Nextcloud Android Client that is present on the mobile device will be lost during the removal process.
General - Enabling Data Limit and Warning
Many mobile network operators charge their subscribers an additional fee when they exceed their data allowance. To avoid these charges, most versions of Android offer some kind of safety measures. These include a Data Limit and Data Warning. The latter notifies you once the specified amount of mobile data has been used up. The former allows you to set a hard limit after which Android will disable mobile data and prevent you from using any more. Also note that you will not be able to use any apps that require an active data connection, such as Internet messengers.
Settings to enable and configure the Data Limit and Warnings can usually be found in the device settings.
Related Issues
Technical Background
Internally, the Nextcloud Android Client uses a database table to keep track of all files. This table has been extended by two columns to accommodate the new two-way sync feature:
internal_two_way_sync_timestampandinternal_two_way_sync_result. The latter of these columns is not relevant for the purposes of this investigation.We use the timestamp to keep track of two things. Firstly, the date when the corresponding file was last synchronised. Secondly, whether the file should be synchronised at all. If the timestamp value of a file contains a valid timestamp greater than
0the file will be scheduled for synchronisation. In this case0means the file has never been synchronised. A value of-1indicates that a file should not be synchronised at all.On a clean install of the Nextcloud client, all records in the
filelisttable will be initialised with a timestamp value of-1. When upgrading from an older version, the database is migrated to fit the new scheme. During this migration, the timestamp column is added to the table and initialised with a default value ofnull. This must be handled carefully when interpreting database values, as improperly handled null values are known to cause a variety of problems. In this case,nullrepresents unknown; the file has never been synchronised and the user hasn't made a decision about whether or not it should be synchronised.In a recent update, a change was made to mitigate null pointer exceptions during file synchronisation. Due to a small oversight, this also meant that all null values would be interpreted as
0. While this fixed the null pointer issues, it unfortunately introduced a new problem.When a user updates to an affected version, the column is filled with
null, which is interpreted as0. After a while, a background worker will look up all files that need to be synchronised in the aforementioned table. Since all files have a0timestamp, it will try to synchronise them all.For technical reasons, we also store the root folder, which contains all other files, in the same database. This folder is represented by a record having
/as its filename. Therefore, all folders and files are scheduled for synchronisation.As it is normally not possible to select the root folder for two-way sync, we have excluded it from the synchronisation process in the patched versions.
Due to the nature of our apps and release channels, rollbacks to previous versions are not possible. Instead, we have released newer versions that fix these issues.
Version History
nullconversion