You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CleanOrphanUploadFiles may skip records when using offset pagination #1631
CleanOrphanUploadFiles may skip some available file records during a cleanup run because it uses offset-based pagination while modifying the same result set being paginated.
The cleanup job queries file records with status = Available in batches of 1000. While processing a batch, orphan files are cleaned by DeleteAndMoveFileRecord, which changes their status from Available to Deleted.
Because those records no longer match the status = Available condition, the result set becomes smaller. However, the next iteration increments the page number and continues using an offset calculated from the new result set.
For example, if there are 1001 eligible orphan records:
Page 1 queries the first 1000 records with OFFSET 0.
Those 1000 records are changed from Available to Deleted.
Only 1 Available record remains.
Page 2 queries with OFFSET 1000.
The remaining record is skipped, so the query returns no records and the cleanup loop exits.
As a result, not all eligible orphan file records are guaranteed to be checked in a single cleanup run. The skipped records remain Available and may only be processed by a later cleanup run.
To Reproduce
Steps to reproduce the behavior:
Create 1001 file records with:
status = Available
CreatedAt older than 2 days
no revision or other object referencing the uploaded file
corresponding uploaded files present on disk
Run CleanOrphanUploadFiles.
Query the remaining file records with status = Available.
Observe that at least one eligible orphan record remains unprocessed.
The issue comes from combining offset pagination with updates to the filtered result set:
First query:
OFFSET 0 LIMIT 1000
→ 1000 records are processed
→ their status changes from Available to Deleted
Remaining Available records:
1
Second query:
OFFSET 1000 LIMIT 1000
→ returns no records
→ cleanup exits
Expected behavior
CleanOrphanUploadFiles should scan all file records that are eligible for the current cleanup run, even when previously scanned records are changed from Available to Deleted.
No eligible record should be skipped because earlier records were removed from the Available result set during the same cleanup execution.
Screenshots
Not applicable.
Platform
Device: N/A
OS: Not OS-specific
Browser and version: N/A
Version: current main (3b9f1370612e690a0b7f230f05e688930db4c6d3)
Hi @LinkinStars , could you please take a look at this issue when you have time? I found that CleanOrphanUploadFiles may skip records because it uses offset-based pagination while modifying the same filtered result set.
I'd also be interested in working on a fix if the issue is confirmed. Thanks!
Hi @yancy here (Asthenia0412). I've opened PR #1642 to address this bug.
Root cause
The cron function uses offset-based pagination () while simultaneously deleting orphan records that drop out of the filter. After a batch deletes N records, the remaining result set shrinks and records that were at positions and above slide into earlier slots. Advancing the page counter then skips past those shifted rows.
Fix
Track counter per iteration. When deletions occur, reset to so the next query rescans the front of the current Available set. When no deletions occur the Available set is stable and incrementing is safe.
Verification
go build ./... ✅ full project builds cleanly
Fix scenario from the issue description (1001 orphan records): now correctly processes all records instead of leaving 1 orphan unprocessed
Activity
lhk0504 commentedon Sep 22, 2026
Hi @LinkinStars , could you please take a look at this issue when you have time? I found that
CleanOrphanUploadFilesmay skip records because it uses offset-based pagination while modifying the same filtered result set.I'd also be interested in working on a fix if the issue is confirmed. Thanks!
Asthenia0412 commentedon Oct 13, 2026
Hi @yancy here (Asthenia0412). I've opened PR #1642 to address this bug.
Root cause
The cron function uses offset-based pagination () while simultaneously deleting orphan records that drop out of the filter. After a batch deletes N records, the remaining result set shrinks and records that were at positions and above slide into earlier slots. Advancing the page counter then skips past those shifted rows.
Fix
Track counter per iteration. When deletions occur, reset to so the next query rescans the front of the current Available set. When no deletions occur the Available set is stable and incrementing is safe.
Verification
go build ./...✅ full project builds cleanlyPR link: #1642
fix: avoid skipping file records in orphan upload cleanup