Skip to content

Nextcloud 20 updates to 21 despite unsupported MySQL 5.x #26470

Description

@caugner

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Steps to reproduce

  1. See MySQL-related warning in Administration / Overview: grafik
  2. See available update to 20.0.9 on same page: grafik
  3. Notice that the Updater wants to update to Nextcloud 21 anyway: grafik

Expected behaviour

  1. Updater performs major update to Nextcloud 21 despite Update notification & Administration Overview advertising patch update to 20.0.9 #26469 - The Updater should update to the same version as advised in the Administration Overview.
  2. The Updater should not update to a major version that is incompatible with the current environment.

Actual behaviour

The Updater updates to the next major version, ignoring the incompatible MySQL version.

Server configuration

Operating system: Ubuntu

Web server: Apache 2.4

Database: 5.7.28-nmm1-log

PHP version: 7.4

Nextcloud version: 20.0.8

Updated from an older Nextcloud/ownCloud or fresh install: ~ ownCloud 5

Where did you install Nextcloud from: Updater?

Signing status:

Signing status
No errors have been found.

List of activated apps:

App list
n/a

Nextcloud configuration:

Config report
n/a

Are you using external storage, if yes which one: No

Are you using encryption: No / Yes (one instance each)

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: Firefox Browser Nightly 89.0a1 (2021-04-08)

Operating system: Windows 10

Logs

Web server error log

Web server error log
(on demand)

Nextcloud log (data/nextcloud.log)

Nextcloud log
(on demand)

Browser log

Browser log
(on demand)

Activity

  1. added
    0. Needs triagePending check for reproducibility or if it fits our roadmap
    on Apr 9, 2021
  2. quizilkend commented on Apr 9, 2021

    @quizilkend

    I experience this as well. Right now my instance is not reachable.

    I am using MariaDB mysql Ver 15.1 Distrib 10.3.27-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 and I'm not sure which version would fit NC21.

  3. caugner commented on Apr 9, 2021

    @caugner
    ContributorAuthor

    I am using MariaDB

    @quizilkend As per the System requirements, Nextcloud 21 supports MySQL 8.0+ or MariaDB 10.2+ (recommended), so you should be fine.

  4. quizilkend commented on Apr 9, 2021

    @quizilkend

    @caugner: Thanks for your reply. Now I have no clue why my instance is not up and running.

    The updater ran flawlessly but occ upgrade hanged silently until the kernel (OOM-killer) killed it.

    Do you have an idea how to further investigate this, to better locate the problem?

  5. caugner commented on Apr 9, 2021

    @caugner
    ContributorAuthor

    @quizilkend Please make sure to clear your browser cache (CTRL+F5). If the problem persists, I would recommend to have a look at the updater.log file in your Nextcloud data directory and open a new bug issue, in which you post the relevant sections from that logfile (i.e. all lines from today).

  6. caugner commented on Apr 9, 2021

    @caugner
    ContributorAuthor

    FYI The good news is that Nextcloud 21 still seems to generally work with MySQL 5.7, even though it's not supported anymore.

  7. andrimont commented on Apr 10, 2021

    @andrimont

    FYI The good news is that Nextcloud 21 still seems to generally work with MySQL 5.7, even though it's not supported anymore.

    @caugner So with -MariaDB-1:10.4.13, I should be fine. But I encounter the same issue as you reported on this thread.
    Did you succeed to install the "stable" 21.0.1 ?
    What is your advice when the install goes wrongly from 20.0.8 to 20.0.9 (announced) but goes to 21.0.1 and then crashes ?

  8. mind-overflow commented on Apr 10, 2021

    @mind-overflow

    FYI The good news is that Nextcloud 21 still seems to generally work with MySQL 5.7, even though it's not supported anymore.

    @caugner So with -MariaDB-1:10.4.13, I should be fine. But I encounter the same issue as you reported on this thread.
    Did you succeed to install the "stable" 21.0.1 ?
    What is your advice when the install goes wrongly from 20.0.8 to 20.0.9 (announced) but goes to 21.0.1 and then crashes ?

    I had literally the same issue. I have written here, on the Nextcloud forums. The update failed and CPU/RAM usage skyrocketed until my server completely died. I'm still in the process of trying to recover access to it.

  9. daCaPo commented on Apr 10, 2021

    @daCaPo

    Upgrade here went well, but I agree that the updater must not update when running with an officially unsuported database (mysql 5.7.33 in my case).

    Furthermore, the settings page now shows a red warning because the PHP memory limit is below 512 MB. An update should not be allowed without explicit warning if the requirements of a new release would not be fulfilled.

  10. caugner commented on Apr 10, 2021

    @caugner
    ContributorAuthor

    Furthermore, the settings page now shows a red warning because the PHP memory limit is below 512 MB.

    This warning is not new to Nextcloud 21. However every update overwrites the .htaccess file, so if you manually added a directive there to increase the memory limit, you will have to add it again. It's a feature, not a bug, to increase stability.

  11. andrimont commented on Apr 10, 2021

    @andrimont

    Upgrade here went well, but I agree that the updater must not update when running with an officially unsuported database (mysql 5.7.33 in my case).

    Did you succeed with 21.0.1 ?
    In my case I used backup and I installed manually 20.0.9. It is roughly ok ; but there is some side effects.
    I still do not get the point of the issue with 20.0.9. > 21.0.1…
    I am very afraid with the install process now

  12. daCaPo commented on Apr 10, 2021

    @daCaPo

    Did you succeed with 21.0.1 ?

    As I can tell so far, yes.

    I am very afraid with the install process now

    So am I, running two more instances whicht I won't touch before this issue is solved.

  13. mind-overflow commented on Apr 10, 2021

    @mind-overflow

    Ok. I unfortunately have no great way to prove this, but here's what I did to recover from the resource-hogging disaster, after forcefully shutting down the machine.

    I disabled apache2 and php7.4-fpm services, to prevent the cronjob from restarting automatically (it actually did, and I had to kill it via htop - but after stopping the services, it stopped respawning).

    Then, I restored 20.0.8 from the /data/updater-xxxxxxx/backups/ folder. I used rsync

    rsync -Aaxv /var/www/nextcloud/data/updater-xxxxxxx/backups/nextcloud-20.0.8.1-xxxxxxxx /var/www/nextcloud/
    

    (be careful with the correct folder names).

    Then, without touching the database, I restarted everything. I visited the Nextcloud URL on the browser, and I logged in. After logging in, I got an internal server error.

    I looked into the Nextcloud log (/var/www/nextcloud/data/nextcloud.log), and I saw an error about missing database tables.

    My intuition was that, probably, the database was updated to the 21.0.1 format. So, to avoid destroying the database, I did this:

    • I went into the nextcloud/updater/ directory.
    • I ran the updater manually.
    sudo -u www-data php updater.phar
    

    It went fairly smoothly, downloaded a bunch of stuff and moved it in place. Then it asked

    Should the "occ upgrade" command be executed? [Y/n]
    

    I said "yes". The update process started, and it showed a bunch of "repair steps". Nice.
    At the end, there was a final step, where the server died on me last time:

    2021-04-10T09:25:32+00:00 Starting code integrity check...
    

    At this point, I launched a parallel SSH terminal and launched htop, to monitor resources usage.

    The Nextcloud cron.php script was launcher once, and taking up about 6GBs of RAM. After 3/4 minutes, it was spawned ANOTHER time (so I had two contemporarily running jobs), and started taking up RAM and CPU. I killed both of them, and RAM usage dropped to 2GB. The update script did not fail however, saying that "integrity check" was passed. (I doubt it, as I literally killed the script that was checking for it - but it doesn't mean it failed either). After that, Nextcloud was responsive again, on latest version 21.0.1.

    2021-04-10T09:30:30+00:00 Finished code integrity check
    2021-04-10T09:30:31+00:00 Update successful
    2021-04-10T09:30:31+00:00 Maintenance mode is kept active
    2021-04-10T09:30:31+00:00 Resetting log level
    

    Skipping integrity check is not a good thing nor a good practice, but it is what it is. It never happened to me in the past that the integrity check would take up so many resources. I am getting no errors in the logs for the moment, so hopefully everything is fine.

  14. szaimen commented on Apr 10, 2021

    @szaimen
    Contributor

    taking up about 6GBs of RAM. After 3/4 minutes, it was spawned ANOTHER time (so I had two contemporarily running jobs), and started taking up RAM and CPU

    Did you see this workaround for high RAM and CPU usage?
    #25761 (comment)

  15. 3 remaining items

  16. changed the title [-]Nextcloud 20.0.8 updates to 21.0.1 despite unsupported MySQL 5.7[/-] [+]Nextcloud 20.0.8 updates to 21.0.1 despite unsupported MySQL 5.x[/+] on Apr 25, 2021
  17. szaimen commented on Jul 2, 2021

    @szaimen
    Contributor

    I think the underlying problem was fixed with nextcloud-releases/updater_server#558

  18. changed the title [-]Nextcloud 20.0.8 updates to 21.0.1 despite unsupported MySQL 5.x[/-] [+]Nextcloud 20 updates to 21 despite unsupported MySQL 5.x[/+] on Jul 2, 2021
  19. caugner commented on Jul 2, 2021

    @caugner
    ContributorAuthor

    @szaimen Afaik that PR only fixed the inconsistency between the advertised update target and the actual update target. It didn't add a check to prevent an update if the target version doesn't support the current environment (here: MySQL 5.6).

  20. NicoHood commented on Oct 9, 2022

    @NicoHood

    I am using Nextcloud 23.0.x, but I am running and old mariadb 10.1.x which should be not supported anymore. However the installation still works. Can anyone tell me what could be broken and if I should better reinstall the whole nextcloud or just keep it running until my provider updates the database version?

  21. szaimen commented on Jan 23, 2023

    @szaimen
    Contributor

    Hi, please update to 24.0.9 or better 25.0.3 and report back if it fixes the issue. Thank you!

    My goal is to add a label like e.g. 25-feedback to this ticket of an up-to-date major Nextcloud version where the bug could be reproduced. However this is not going to work without your help. So thanks for all your effort!

    If you don't manage to reproduce the issue in time and the issue gets closed but you can reproduce the issue afterwards, feel free to create a new bug report with up-to-date information by following this link: https://github.com/nextcloud/server/issues/new?assignees=&labels=bug%2C0.+Needs+triage&template=BUG_REPORT.yml&title=%5BBug%5D%3A+

  22. added
    0. Needs triagePending check for reproducibility or if it fits our roadmap
    and removed
    1. to developAccepted and waiting to be taken care of
    on Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions