Skip to content

[primary storage on Openstack swift] Update to 12.0.5 / 13RC breaks Nextcloud #8041

Description

@harryhb

Issue logged after discussion on support: https://help.nextcloud.com/t/broken-after-update-to-12-0-5/26300

Steps to reproduce

  1. Upgrade from 12.0.4 tot 12.0.5 / 13RC through admin interface

Expected behaviour

Upgrade script runs; through web interface the upgrade can be finalized and system is back up and running.

Actual behaviour

  • The "button" on the web interface cannot be activated => Finish upgrade through CLI
  • The site login screen can be reached, but login fails (buttons cannot be activated)

Server configuration

Operating system: Debian stable

Web server: Apache/2.4.25 (Debian)

Database: mariadb Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

PHP version: PHP 7.0.27-0+deb9u1

Nextcloud version: 12.0.5 / 13RC

Updated from an older Nextcloud/ownCloud or fresh install: Updated from 12.0.4

Where did you install Nextcloud from: Through admin interface

Signing status: Not possible to login

List of activated apps:

App list Enabled: - activity: 2.5.2 - admin_audit: 1.2.0 - bruteforcesettings: 1.0.3 - calendar: 1.5.7 - caniupdate: 0.1.1 - comments: 1.2.0 - contacts: 2.0.1 - dav: 1.3.1 - federatedfilesharing: 1.2.0 - federation: 1.2.0 - files: 1.7.2 - files_downloadactivity: 1.1.1 - files_pdfviewer: 1.1.1 - files_sharing: 1.4.0 - files_texteditor: 2.4.1 - files_trashbin: 1.2.0 - files_versions: 1.5.0 - files_videoplayer: 1.1.0 - firstrunwizard: 2.1 - gallery: 17.0.0 - logreader: 2.0.0 - lookup_server_connector: 1.0.0 - mail: 0.7.8 - metadata: 0.6.0 - nextcloud_announcements: 1.1 - notifications: 2.0.0 - oauth2: 1.0.5 - ocsms: 1.12.2 - passman: 2.1.4 - password_policy: 1.2.2 - provisioning_api: 1.2.0 - qownnotesapi: 17.5.0 - serverinfo: 1.2.0 - sharebymail: 1.2.0 - spreed: 2.0.2 - survey_client: 1.0.0 - systemtags: 1.2.0 - tasks: 0.9.5 - twofactor_backupcodes: 1.1.1 - updatenotification: 1.2.0 - workflowengine: 1.2.0

Nextcloud configuration:

Config report

{
"system": {
"objectstore": {
"class": "OC\Files\ObjectStore\Swift",
"arguments": {
"username": "xx",
"password": "REMOVED SENSITIVE VALUE",
"bucket": "nextcloud_swift",
"autocreate": true,
"region": "xx",
"url": "https://identity.stack.cloudvps.com/v2.0",
"tenantName": "xx",
"serviceName": "swift"
}
},
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"DOMAIN"
],
"datadirectory": "xx",
"overwrite.cli.url": "xx",
"dbtype": "mysql",
"version": "xx",
"dbname": "xx",
"dbhost": "localhost",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"installed": true,
"forcessl": true,
"logtimezone": "UTC",
"trashbin_retention_obligation": "auto",
"mail_from_address": "info",
"mail_smtpmode": "smtp",
"mail_domain": "xx",
"mail_smtpsecure": "tls",
"mail_smtpauthtype": "xx",
"mail_smtpauth": 1,
"mail_smtphost": "xx",
"mail_smtpport": "xx",
"mail_smtpname": "REMOVED SENSITIVE VALUE",
"mail_smtppassword": "REMOVED SENSITIVE VALUE",
"app.mail.smtp.timeout": 10,
"memcache.local": "\OC\Memcache\Redis",
"filelocking.enabled": "true",
"memcache.distributed": "\OC\Memcache\Redis",
"memcache.locking": "\OC\Memcache\Redis",
"redis": {
"host": "localhost",
"port": 6379,
"timeout": 0,
"dbindex": 0
},
"appstore.experimental.enabled": true,
"maintenance": false,
"theme": "",
"loglevel": 0,
"updater.secret": "REMOVED SENSITIVE VALUE",
"instanceid": "xx"
}
}

Are you using external storage, if yes which one: Primary storage on Openstack/Swift

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Firefox & Chromium

Operating system: Debian testing

Logs

Web server error log

Web server error log No errors

Nextcloud log (data/nextcloud.log)

Nextcloud log

Browser log

Browser log ReferenceError: OC is not defined

Activity

  1. Arvi89 commented on Jan 26, 2018

    @Arvi89

    Any news? This is kind of a big deal.

  2. harryhb commented on Jan 26, 2018

    @harryhb
    Author

    I reverted back to 12.0.4 as this bug is a deal breaker for nextcloud usage.

  3. Arvi89 commented on Jan 28, 2018

    @Arvi89

    I find it amazing to not have an answer from anyone (even just to ask for more information). It's a serious issue, and it's kind of ignored. A minor update just completely broke the nextcloud installation, but hey, no big deal, this affects only few people, no worry.

  4. mduller commented on Jan 28, 2018

    @mduller

    I am actually holding off performing the upgrade because of this issue that "affects only few people" -- I currently don't have the luxury of spending time on rolling back if something goes wrong (I am in charge of 4 Nextcloud systems). Would like to see some insights in this ticket to inspire confidence to perform the update. That confidence is currently missing.

  5. Schmuuu commented on Jan 28, 2018

    @Schmuuu

    I find it amazing to not have an answer from anyone (even just to ask for more information). It's a serious issue, and it's kind of ignored. A minor update just completely broke the nextcloud installation, but hey, no big deal, this affects only few people, no worry.

    Hi Arvi,

    I totally understand your frustration about your broken NC server due to the minor update to 12.0.5.
    Let's put some focus on this issue.

    @ChristophWurst @MorrisJobke @jospoortvliet @rullzer Who can help? Quite a few users broke their installation by updating to 12.0.5
    The community couldn't help so far. Some insights of developers is needed.

    Please advise.

  6. hitam4450 commented on Jan 28, 2018

    @hitam4450

    Updating the server instance is always a total mess! Minor or major update is always trouble !
    Plan enough time for manual update, that is the only way to (more - or less ) safe to update!

  7. rullzer commented on Jan 28, 2018

    @rullzer
    Member

    I can't find the bug I think this is...

    Anyway.

    Could you try:

    1. Remove the css and js folder from your appdata_* folder
    2. Run: ./occ files:scan-app-data
    3. Do a force refresh in your browser ctrl + F5

    @skjnldsv @juliushaertl I think #8078 would help here as well.

  8. Arvi89 commented on Jan 28, 2018

    @Arvi89

    mm I don't have any appdata folder, where is it supposed to be?

  9. rullzer commented on Jan 28, 2018

    @rullzer
    Member

    @Arvi89 in your data folder. It is data/appdata_<instance id>.

  10. Arvi89 commented on Jan 28, 2018

    @Arvi89

    Yeah, I don't have that folder.
    (I use openstack as my main storage)

  11. rullzer commented on Jan 28, 2018

    @rullzer
    Member

    Ah right then your data folder is indeed empty I guess.

    I can try to come up with a SQL query... because if you just purge it from the DB on object store it will just create new items :)

  12. rullzer commented on Jan 28, 2018

    @rullzer
    Member
    DELETE
    FROM oc_filecache
    WHERE path LIKE 'appdata%/js/%'
    OR path LIKE 'appdata%/css/%';

    Try that. Of course make sure you have a backup and all that ;)

  13. Arvi89 commented on Jan 28, 2018

    @Arvi89

    No luck. The query was executed properly (203 rows affected), but it doesn't work better.

  14. 23 remaining items

  15. Arvi89 commented on Feb 7, 2018

    @Arvi89

    Does somebody have a working swift setup for me to test against? Because the docker images I could find did not work.

    What do you mean, you guys seriously still haven't installed a nextcloud instance with swift, really???

  16. tdttester commented on Feb 7, 2018

    @tdttester

    They are releasing a product with a feature that they can't test whether it works or not. It's amazing!

  17. tessus commented on Feb 7, 2018

    @tessus

    @Schmuuu @skjnldsv why the thumbs down? it's the truth, otherwise this would have been a showstopper for a release, or at least I hope so.
    The only dev in this discussion said that he can't test/reproduce, because the Docker images don't work.
    If the CI pipeline had sufficient tests, the commit that broke nextcloud would've never been merged.

    So nothing what @tdttester said is wrong.

  18. rullzer commented on Feb 7, 2018

    @rullzer
    Member

    Lets keep it civil people. We are constantly updating and extending our test cases. This test matrix is already big and getting bigger every day.

    We are in the process of setting up automated tests against swift right now. But this takes time and resources which are limited.

    Hopefully we have a fix for you to test soon. But comments like that don't make us work any faster. The opposite is more likely.

  19. tessus commented on Feb 7, 2018

    @tessus

    Please note that I did not attack anybody personally. I also did not use any swear words, nor harsh language. Don't deflect by calling the truth uncivil.

    I know that it is hard to have test cases for every single edge case, but this doesn't change the fact that my comment above was correct.

  20. skjnldsv commented on Feb 7, 2018

    @skjnldsv
    Member

    @tessus it's called diplomacy. Saying things sarcastically has never improved the dialogue between people.

    E.g: three sentences, one same outcome

    1. Maybe if you had done your work correctly and implemented the proper testing, you wouldn't be there
    2. Maybe we should consider adding some more tests for the next time so things like that doesn't occur again
    3. Releasing a product with a feature that they can't test whether it works or not. It's amazing!

    Which one do you prefer? I made my choice :)

  21. tessus commented on Feb 7, 2018

    @tessus

    @skjnldsv ok, my bad. I'm blunt and I expect others to be blunt with me as well. for me, all 3 sentences have the same meaning: I screwed up and thus have to fix it.
    I would never take any of it personally or think "oops, that sounded harsh and hurt my feelings".

    But I agree with you, the second one may "sound" nicer to a lot of people. Personally I don't put value in how something sounds, but the context.

    Disclaimer: I have a genuine distaste for sugar-coating and making things sound nice for reasons of diplomacy and political correctness. I'm working on it, but it may take a while... ;-)

    Anyway, I don't want to derail this ticket any further. I just wanted to make my position clear.
    (I won't add any more comments to this ticket.)

    P.S.: I also love sarcasm.

  22. skjnldsv commented on Feb 7, 2018

    @skjnldsv
    Member

    @tessus no worries. But yes, it's not about sugar coating things, it's about telling the other person you're with him to fix the issue (which is productive) and not focusing on the issue itself (which is counter-productive) :)

  23. rullzer commented on Feb 7, 2018

    @rullzer
    Member

    I think this is the same issue as #8233

    The fix should be trivial to apply on 13 and 12. Please test. I'll make sure it is backported if it fixes it.

  24. Arvi89 commented on Feb 7, 2018

    @Arvi89

    Yep, this is working. Adding the return statement fixed my problem! Thank you!

  25. rullzer commented on Feb 8, 2018

    @rullzer
    Member

    Merged and will be shipped with 12.0.6, 13.0.1 and 14

    Master: #8233
    Stable 13: #8250
    Stable 12: #8251

    Grab the patch from your version. It is trivial to apply.

    Also see #8235, this should get more swift test coverage to the server so this happening again will be a lot less likely.

  26. leoverdura commented on Feb 9, 2018

    @leoverdura

    I just manually updated to Nextcloud 13. Procedure like described in https://docs.nextcloud.com/server/13/admin_manual/maintenance/manual_upgrade.html
    That helped me out. It works all fine again as far as I can see.

  27. benjauger commented on Feb 22, 2018

    @benjauger

    Same issue, for upgrade N12 -> 13, fixed with #8250
    Thanks @rullzer !

  28. linuxipho commented on Feb 22, 2018

    @linuxipho

    #8250 works perfectly for 12 to 13 upgrade with Swift ObjectStore.
    Thanks again @rullzer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmapbug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions