Repository navigation
After deleting image, the preview files remain forever #11574
Description
Activity
GitMate.io thinks possibly related issues are #5558 (After deleting an (image) file, the preview files remain forever), #9523 (A file completely disappears after deleting from a group folder), #8241 (image), #2413 (Previews for empty files broken), and #3041 (Deleting a preview folder stops writing previews to disk).
Can you check if you have an entry in your background jobs: OC\Preview\BackgroundCleanupJob?
Also note that if you use ajax cron it will only delete 10 at a time (to not run into timeouts).
I am using a Linux cron job, not Ajax cron.
Sorry; how do I access OC\Preview\BackgroundCleanupJob?
In the logs I found the following which is related to background Cleanup Jobs:
{"reqId":"0br9L4RsCRwi9HJhnVr9","level":1,"time":"2018-06-08T10:32:51+02:00","remoteAddr":"xxx","user":"--","app":"updater","method":"GET","url":"\/core\/ajax\/update.php?requesttoken=SvT6GXpQphnGbl9Mg9WdNOUBmJ35bJJ5GLpgNhsC%2FQk%3D%3AJ6yiexIZkHu8I24G6L6sYLBY0tyQAuU6XdE5cllluD0%3D","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.2.1"} {"reqId":"JiMmzSdZG0ya4yWgnY5p","level":1,"time":"2018-06-12T07:45:32+02:00","remoteAddr":"","user":"--","app":"updater","method":"--","url":"--","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"--","version":"13.0.3.2"} {"reqId":"Vuip1twG4c8HRKSb8r1M","level":1,"time":"2018-08-30T18:40:17+02:00","remoteAddr":"","user":"--","app":"updater","method":"--","url":"--","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"--","version":"13.0.4.0"} {"reqId":"3N5R2TG63uE5RtzH9Hh1","level":1,"time":"2018-08-30T18:40:48+02:00","remoteAddr":"xxxxxx","user":"--","app":"updater","method":"GET","url":"\/core\/ajax\/update.php?requesttoken=Y9%2FT7HxGSiDjwrAVSLSFWrJQPYx4Lr9Dd44Qy4F6hmc%3D%3AULXl3R8pPGKRuoRsC4G1PcZ7CcVNVsUkJtdyr9VL1gU%3D","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.2 Safari\/605.1.15","version":"13.0.4.0"}Q: What would actually happen if I just delete the entire preview folder? Would the preview generator understand that some needed previews are missing and recreate them?
AH sorry. Check the oc_jobs table.
No it won'tyou need to rescan your appdata folder with the occ command
Hope I got that right:
select * from oc_jobs where class="OC\Preview\BackgroundCleanupJob";does not return anything.I can confirm this is still an issue in NC15.0.8
I have experienced the same issue in my installation of NextCloud on a nextcloudbox running nextcloudpi. I believe the problem lied with the preview generator app. When I realized the previews were eating up a huge amound of space I ended up deleting the app and manually removed the offending data.
@Lolaum since you are using NCP it is configured now to not make unnecessary previews so it should be faster and take up less space.
Previews are generated one way or another (either by the pregen app or through browsing). Whatever the case may be when you delete the files the previews remain both in the previews folder and in the database and they are never pruned.
This issue was the one that came up in searches, so someone please link to a more active issue if there is one. I had found that updating previews was taking a very long time, so I moved the appdata/preview folder to one side, rescanned my files, and then reran the preview generator. It took a while, but got there soon enough. The newly generated preview folder contains 61 MB of data. The one I moved contained 1.1 GB of data. So it seems that the preview folder bloats rather than keeping itself clear of previews of files that no longer exist.
This issue is obviously related to #5558, which was closed for Nextcloud 12. I run Nextcloud 13.0.6 and the issue exists for me. Can anyone confirm?
Steps to reproduce
(5. Wait for trashbin_retention_obligation is exceeded)
Expected behaviour
The cached previews should be deleted at some point in time...
Actual behaviour
The cached previews are NOT deleted, in my case I have 13GB of preview images of different sizes
Server configuration
Operating system:
Debian
Web server:
Nginx
Database:
Maria
PHP version:
7.2
Nextcloud version: (see Nextcloud admin page)
13.0.6
Updated from an older Nextcloud/ownCloud or fresh install:
Updated; but previews were generated on NC 13
List of activated apps:
Details
``` - activity: 2.6.1 - admin_audit: 1.3.0 - admin_notifications: 1.0.1 - apporder: 0.5.0 - bookmarks: 0.12.2 - bookmarks_fulltextsearch: 1.0.0 - bruteforcesettings: 1.1.0 - calendar: 1.6.1 - caniupdate: 0.1.3 - contacts: 2.1.5 - dav: 1.4.7 - federatedfilesharing: 1.3.1 - files: 1.8.0 - files_fulltextsearch: 1.0.0 - files_fulltextsearch_tesseract: 1.0.0 - files_pdfviewer: 1.2.1 - files_rightclick: 0.8.4 - files_sharing: 1.5.0 - files_texteditor: 2.5.1 - files_trashbin: 1.3.0 - files_versions: 1.6.0 - files_videoplayer: 1.2.0 - fulltextsearch: 1.0.0 - fulltextsearch_elasticsearch: 1.0.0 - gallery: 18.0.0 - impersonate: 1.0.4 - logreader: 2.0.0 - lookup_server_connector: 1.1.0 - metadata: 0.7.0 - nextcloud_announcements: 1.2.0 - notifications: 2.1.2 - oauth2: 1.1.1 - password_policy: 1.3.0 - previewgenerator: 1.1.0 - provisioning_api: 1.3.0 - serverinfo: 1.3.0 - sharerenamer: 1.5.0 - twofactor_backupcodes: 1.2.3 - twofactor_totp: 1.4.1 - updatenotification: 1.3.0 - workflowengine: 1.3.0 ```Nextcloud configuration:
Details
``` { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "xxxxxx", "xxxxxx" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "xxxxxxxxx", "dbtype": "mysql", "version": "13.0.6.1", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "memcache.local": "\\OC\\Memcache\\APCu", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "maintenance": false, "logtimezone": "Europe\/Berlin", "loglevel": 0, "theme": "", "trashbin_retention_obligation": "auto, 7", "updater.release.channel": "production", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpsecure": "tls", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***" } ```