Repository navigation
Database conversion: hangs forever #173
Description
Steps to reproduce
- Create an empty PostgreSQL database
- Perform a database conversion:
./occ db:convert-type --password <password> --all-apps --clear-schema pgsql <username> <hostname> <database>
Expected behaviour
A working database conversion.
Actual behaviour
The conversion hangs forever at oc_cards:
[snipped]
oc_activity
4948/4948 [============================] 100%
oc_activity_mq
0/0 [============================] 0%
oc_addressbooks
8/8 [============================] 100%
oc_cards
330/1162 [=======>--------------------] 28%
Server configuration
Operating system: Arch Linux
Web server: Nginx
Database: SQLite
PHP version: 7.0.7
Nextcloud version: 9.0.50
Updated from an older Nextcloud/ownCloud or fresh install: Updated, starting from ownCloud 3.0
Where did you install Nextcloud from: archive
Signing status (Nextcloud 9.0 and above):
No errors have been found.
List of activated apps:
Enabled:
- activity: 2.2.1
- admin_audit: 1.0.0
- calendar: 1.2.2
- comments: 0.2
- contacts: 1.3.1.0
- dav: 0.1.6
- federatedfilesharing: 0.1.0
- federation: 0.0.4
- files: 1.4.4
- files_pdfviewer: 0.8.1
- files_sharing: 0.9.1
- files_trashbin: 0.8.0
- files_versions: 1.2.0
- files_videoplayer: 0.9.8
- firstrunwizard: 1.1
- gallery: 14.5.0
- notes: true
- notifications: 0.2.3
- provisioning_api: 0.4.1
- systemtags: 0.2
- tasks: true
- templateeditor: 0.1
- updatenotification: 0.1.0
Disabled:
- encryption
- external
- files_external
- files_texteditor
- user_external
- user_ldap
Are you using encryption: no
Activity
MorrisJobke commentedon Jun 20, 2016
Is there anything in the log file?
nblock commentedon Jun 20, 2016
Nope
nblock commentedon Jun 20, 2016
The postgres log has something interesting:
It might be related to owncloud/core#24065. The converter crashes in the referenced issue. It hangs forever in my case.
nblock commentedon Jun 21, 2016
I deleted problematic rows (those had large contact photos) and now the converter hits this entry:
The good thing is: the converter now crashes and does no longer hang.
nickvergessen commentedon Sep 13, 2016
Crashes means it throws an exception?
nblock commentedon Sep 14, 2016
@nickvergessen Yes
nblock commentedon Nov 5, 2016
I just tested the migration from sqlite to postgresql on Nextcloud 10.0.1 and it has the same issues as described above. Is there anything else I can provide?
gjedeer commentedon Dec 22, 2016
I'm getting the same error when converting oc_cards from sqlite to pgsql. It looks like row type information is not being preserved in ConvertType.php in copyTable() where rows are being read from one database and written to the other. setParameter() can optionally take a 3rd argument, the type, but it's not being set.
But that's all I know, deducted this from stack traces, I know nothing about Doctrine or nextCloud code base and internet searches weren't helpful in figuring out how to retrieve the type information from source database. So no patch from me.
Having type information for binary columns would allow the database abstraction layer to properly escape the values before inserting them.
Stack trace: https://gist.github.com/gjedeer/603f9f882e272ac7ae9eb0d614a33fcb
MorrisJobke commentedon Jan 25, 2017
@icewind1991 could you have a look?
pkel commentedon Mar 22, 2017
I triggered the same behaviour while migrating from maria to postgres.
nickvergessen commentedon Mar 26, 2017
I identified the problem, it's rather arbitrary that it happens with contacts.
It's actually related to the clob column we are using. Any table with such a column and the content
\,causes the problem.gjedeer commentedon Apr 4, 2017
@nickvergessen thank you a lot! Hopefully I'll now be able to move to Postgres.