Skip to content

Ensure every file is scanned even when Worker is killed - #13262

Merged
alperozturk96 merged 8 commits into
masterfrom
make-filesync-worker-persistent-new-idea
Aug 1, 2024
Merged

Ensure every file is scanned even when Worker is killed#13262
alperozturk96 merged 8 commits into
masterfrom
make-filesync-worker-persistent-new-idea

Conversation

@JonasMayerDev

@JonasMayerDev JonasMayerDev commented Jul 10, 2024 •
edited
Loading

Copy link
Copy Markdown
Collaborator

As this PR failed because it produces too much overhead, a new approach consists of just shuffling the files before scanning to prevent a bias where files "further at the back" get not scanned in large folders. It is probably not the prettiest solution but since this problem is only theoretical (no known issues) and an edge case with the new improvements to auto upload this should be enough to prevent bias.

Also Fixes #13008 and fixes #13078
Finally finishes #12954

Additionally, this PR includes:

  1. Fixes a bug where the lastScannedTimestamp is not saved to the database -> Gaining a lot of performance
  2. Make sure the files are not only scanned but also uploaded
  3. Increase the FileSyncWorker execution interval if battery is low or conditions are not met (no internet)

To test, please test the whole auto upload feature, since there are a lot of small changes done that could affect the whole feature.

  • Tests written, or not not needed

@JonasMayerDev JonasMayerDev mentioned this pull request Jul 10, 2024
Closed
1 task
@JonasMayerDev JonasMayerDev reopened this Jul 10, 2024
@JonasMayerDev
JonasMayerDev marked this pull request as draft July 10, 2024 16:44
@JonasMayerDev JonasMayerDev mentioned this pull request Jul 10, 2024
Closed
5 tasks done
@JonasMayerDev JonasMayerDev linked an issue Jul 11, 2024 that may be closed by this pull request
Remove foreground worker for auto upload #12954
Closed
5 tasks done
@JonasMayerDev
JonasMayerDev force-pushed the make-filesync-worker-persistent-new-idea branch from 1a9c5a5 to 397077f Compare July 30, 2024 16:41
@JonasMayerDev
JonasMayerDev force-pushed the make-filesync-worker-persistent-new-idea branch from 9854baa to b9295a6 Compare July 31, 2024 11:49
@JonasMayerDev
JonasMayerDev marked this pull request as ready for review July 31, 2024 11:56
@github-actions

github-actions Bot commented Jul 31, 2024

Copy link
Copy Markdown

APK file: https://www.kaminsky.me/nc-dev/android-artifacts/13262.apk

qrcode

To test this change/fix you can simply download above APK file and install and test it in parallel to your existing Nextcloud app.

@github-actions

github-actions Bot commented Jul 31, 2024

Copy link
Copy Markdown

Codacy

Lint

TypemasterPR
Warnings6262
Errors33

SpotBugs

CategoryBaseNew
Bad practice6363
Correctness6363
Dodgy code306306
Experimental11
Internationalization77
Multithreaded correctness66
Performance5252
Security1818
Total516516

alperozturk96
alperozturk96 approved these changes Aug 1, 2024
View reviewed changes

@alperozturk96 alperozturk96 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested

@alperozturk96
alperozturk96 merged commit 53f2ded into master Aug 1, 2024
@AndyScherzinger AndyScherzinger added this to the Nextcloud App 3.30.0 milestone Aug 1, 2024
@github-actions

github-actions Bot commented Aug 6, 2024

Copy link
Copy Markdown

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alperozturk96 alperozturk96 alperozturk96 approved these changes
@ZetaTom ZetaTom Awaiting requested review from ZetaTom
@tobiasKaminsky tobiasKaminsky Awaiting requested review from tobiasKaminsky

Assignees

No one assigned

Labels

feature: auto upload feedback-requested

Projects

None yet

Milestone

Nextcloud App 3.30.0

4 participants

@JonasMayerDev @alperozturk96 @AndyScherzinger @joshtrichards