After deleting an (image) file, the preview files remain forever

Steps to reproduce

Upload any image file (i.e. via the web interface)
Open it / look at it, so that preview files are being created in appdata_appid/preview
Delete the file via the web interface
Empty Nextcloud's trash bin
Check the preview files on your HDD. You will still find the previews of the image you have just deleted.

I recorded my screen to show you what is going on.
In the video I'm going to upload a picture, then I'm going to look at the previews so that those are created and then I delete the file again. Simultaniously I keep an eye on the appdata_appid/preview folder. Watch the preview folder/file dates, then you'll see what I mean.
https://goo.gl/photos/5u1PCSPWU61we8EM8

Expected behaviour

All preview files of deleted images should be deleted as well.

Actual behaviour

Preview files are not being deleted.
Well, let's put it this way... It seems like they are being deleted, but at the same time, some new preview files are being created.

Server configuration

Operating system: Ubuntu 16.04

Web server: Apache

Database: MySQL

PHP version: 7.0

Nextcloud version: 12.0

Updated from an older Nextcloud/ownCloud or fresh install: Update from 11.0.3

Where did you install Nextcloud from: Manual installation via zipped files

We do delete the previews when you remove the file. However it seems some are recreated when you view the file in the trashbin. And then when you empty that they are not deleted.

I'll look into this

@icewind1991

Aaah so it seems to happen because we don't emit the normal pre/post delete hooks for the trashbin... @icewind1991 any particular reason for that?

However it seems some are recreated when you view the file in the trashbin. And then when you empty that they are not deleted.

That is correct. That's exactly whats happening.

Meanwhile I have written a little PHP script, which deletes all the unused preview files. (In fact, as a first step, it does not delete the files but moves it into a different directory - safety reasons, as I was not sure if the script is doing what it's supposed to :) )

I want to share it with you, but i need to warn you!
My PHP or programming knowledges are very limited. I just wrote it very quickly for my personal use to get rid of the gigabytes of unused previews I have. So this little script is only there to fulfill its purpose. There was no focus on efficiency or security.

I run the script outside the web directory using the command line.
It only works with MySQL databases. Successfully tested it with NC 12.0.

<?php

$NC_system_path = "/var/www/html/NC/"; #needs to end with /
$NC_files_path = "/home/user/NC_data/"; #needs to end with /
$NC_appdata_path = "appdata_o9854jfqxnmj/"; #needs to end with /
$NC_newpath = "/home/user/NC_deleted_previews/"; #needs to end with /
#Instead of deleting the preview files, this script moves the preview folders/files to $NC_newpath.... you never know ;-)

#If not intended, there is nothing to edit below this line

$NC_full_appdata_path = $NC_files_path.$NC_appdata_path;

#To get DB-Information
include($NC_system_path.'config/config.php');

$pdo = new PDO('mysql:host=localhost;dbname='.$CONFIG['dbname'],$CONFIG['dbuser'],$CONFIG['dbpassword']);
$sql = "SELECT `fileid` FROM `oc_filecache`;";

foreach ($pdo->query($sql) as $row) {
	$files_in_db[]=$row['fileid'];
}


$files_in_dir = scandir($NC_full_appdata_path.'preview');

$unused_dirs = array_diff($files_in_dir, $files_in_db);

foreach ($unused_dirs as $row) {
	
	if ($row != "." and $row != "..") {
		rename($NC_full_appdata_path.'preview/'.$row,$NC_newpath.$row);
		echo "Moved ".$NC_full_appdata_path.'preview/'.$row."\n    to ".$NC_newpath.$row."\n";
	}
}


?>

I'd really appreciate if someone can take care of this bug

@icewind1991

Aaah so it seems to happen because we don't emit the normal pre/post delete hooks for the trashbin... @icewind1991 any particular reason for that?

Ping

maybe also @schiessle

@icewind1991

I had a chat with @icewind1991
The main reason is it is outside the user files. which makes sense else we'd run a serious risk of running into loops etc.

Actually if we would use the Node API we'd have proper hooks to listen to.

So long story short. We should kill the view in files_trashbin (and all other places actually :P). But that is not a quick or easy fix.

So long story short. We should kill the view in files_trashbin (and all other places actually :P).

Agree, but for now it is probably easier to just listen to the pre-/post-delete hook of the trashbin to clean up previews: https://github.com/nextcloud/server/blob/master/apps/files_trashbin/lib/Trashbin.php#L530

Ah right. Mmmm might require me do do some nasty hacks. but I'll look into a quick fix.

Thank you for taking care of this.

Fix in #5852

@lukasb314

@lukasb314 Thank you for your script above!!! It just works!

If you delete a folder with lots of images and PHP runs out of memory processing the delete hooks, is there a process to clean the orphaned previews up asynchronously?

@rullzer

If you delete a folder with lots of images and PHP runs out of memory processing the delete hooks, is there a process to clean the orphaned previews up asynchronously?

This is usually done in a background job. cc @rullzer

We run a background job to clean up previews that do not have a corresponding entry in the filecache.

So long story short. We should kill the view in files_trashbin (and all other places actually :P).

Agree, but for now it is probably easier to just listen to the pre-/post-delete hook of the trashbin to clean up previews: https://github.com/nextcloud/server/blob/master/apps/files_trashbin/lib/Trashbin.php#L530

But what if I disable the trash bin? Previews are not deleted now

Navigation Menu

Repository navigation

After deleting an (image) file, the preview files remain forever #5558

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

After deleting an (image) file, the preview files remain forever #5558

Description

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Activity

rullzer commented on Jun 30, 2017

rullzer commented on Jun 30, 2017

lukasb314 commented on Jun 30, 2017

lukasb314 commented on Jul 5, 2017

MorrisJobke commented on Jul 18, 2017

rullzer commented on Jul 19, 2017

schiessle commented on Jul 20, 2017

rullzer commented on Jul 21, 2017

lukasb314 commented on Jul 23, 2017

rullzer commented on Jul 24, 2017

alexandroskr77 commented on Oct 1, 2017

someonestolemyusername commented on Aug 24, 2018

MorrisJobke commented on Aug 24, 2018

rullzer commented on Aug 24, 2018

novikov1337danil commented on Jan 4, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions