Repository navigation
OpenStack Swift external storage broke after upgrade to 14.0.0 #11264
Description
Activity
Also, when clicking on the folder (in pink) the following message appears:
There was an error with message: Empty response from the server. Do you want to review mount point config in personal settings page?
If one clicks on the "OK" button, it simply sends me back to the folder list and not to the "mount point config in personal settings page" as expected.
I have also confirmed that the Nextcloud server can reach the Swift cluster just fine.
None of the above bot-suggested tickets appear to be relevant.
Same issue for me, i've got this message from command line updater :
Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade Set log level to debug Updating database schema Updated database Checking for update of app activity in appstore An unhandled exception has been thrown: Error: Call to undefined method GuzzleHttp\Client::request() in /var/www/nextcloud/3rdparty/php-opencloud/openstack/src/Common/Api/OperatorTrait.php:118 Stack trace: #0 /var/www/nextcloud/3rdparty/php-opencloud/openstack/src/Common/Api/OperatorTrait.php(126): OpenStack\Common\Service\AbstractService->sendRequest(Object(OpenStack\Common\Api\Operation), Array) #1 /var/www/nextcloud/3rdparty/php-opencloud/openstack/src/Identity/v2/Service.php(52): OpenStack\Common\Service\AbstractService->execute(Array, Array) #2 /var/www/nextcloud/lib/private/Files/ObjectStore/SwiftFactory.php(147): OpenStack\Identity\v2\Service->generateToken(Array) #3 /var/www/nextcloud/lib/private/Files/ObjectStore/SwiftFactory.php(117): OC\Files\ObjectStore\SwiftFactory->auth(Object(OpenStack\Identity\v2\Service), '8WcCwwSpbKpm@ht...') #4 /var/www/nextcloud/lib/private/Files/ObjectStore/SwiftFactory.php(189): OC\Files\ObjectStore\SwiftFactory->getClient() #5 /var/www/nextcloud/lib/private/Files/ObjectStore/SwiftFactory.php(177): OC\Files\ObjectStore\SwiftFactory->createContainer() #6 /var/www/nextcloud/lib/private/Files/ObjectStore/Swift.php(57): OC\Files\ObjectStore\SwiftFactory->getContainer() #7 /var/www/nextcloud/lib/private/Files/ObjectStore/Swift.php(89): OC\Files\ObjectStore\Swift->getContainer() #8 /var/www/nextcloud/lib/private/Files/ObjectStore/ObjectStoreStorage.php(276): OC\Files\ObjectStore\Swift->readObject('urn:oid:336') #9 /var/www/nextcloud/lib/private/Files/Storage/Common.php(191): OC\Files\ObjectStore\ObjectStoreStorage->fopen('appdata_oc0lh55...', 'r') #10 /var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(244): OC\Files\Storage\Common->file_get_contents('appdata_oc0lh55...') #11 /var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php(266): OC\Files\Storage\Wrapper\Wrapper->file_get_contents('appdata_oc0lh55...') #12 /var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(244): OC\Files\Storage\Wrapper\Availability->file_get_contents('appdata_oc0lh55...') #13 /var/www/nextcloud/lib/private/Files/View.php(1146): OC\Files\Storage\Wrapper\Wrapper->file_get_contents('appdata_oc0lh55...') #14 /var/www/nextcloud/lib/private/Files/View.php(588): OC\Files\View->basicOperation('file_get_conten...', '/appdata_oc0lh5...', Array) #15 /var/www/nextcloud/lib/private/Files/Node/File.php(52): OC\Files\View->file_get_contents('/appdata_oc0lh5...') #16 /var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFile.php(88): OC\Files\Node\File->getContent() #17 /var/www/nextcloud/lib/private/App/AppStore/Fetcher/Fetcher.php(146): OC\Files\SimpleFS\SimpleFile->getContent() #18 /var/www/nextcloud/lib/private/Installer.php(385): OC\App\AppStore\Fetcher\Fetcher->get() #19 /var/www/nextcloud/lib/private/Updater.php(445): OC\Installer->isUpdateAvailable('activity') #20 /var/www/nextcloud/lib/private/Updater.php(256): OC\Updater->upgradeAppStoreApps(Array) #21 /var/www/nextcloud/lib/private/Updater.php(126): OC\Updater->doUpgrade('14.0.3.0', '13.0.6.1') #22 /var/www/nextcloud/core/Command/Upgrade.php(262): OC\Updater->upgrade() #23 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(251): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #24 /var/www/nextcloud/3rdparty/symfony/console/Application.php(946): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #25 /var/www/nextcloud/3rdparty/symfony/console/Application.php(248): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #26 /var/www/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #27 /var/www/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #28 /var/www/nextcloud/console.php(96): OC\Console\Application->run() #29 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')Any news for this ?
I also encountered this issue today and got the following log message:
{"reqId":"<redacted>","level":1,"time":"November 21, 2018 21:38:20","remoteAddr":"<redacted>","user":"felix","app":"no app in context","method":"PUT","url":"\/apps\/files_external\/globalstorages\/1","message":"External storage not available: Storage unauthorized. Scope has to be defined for V3 requests","userAgent":"<redacted>","version":"14.0.3.0"}Searching the last part of the message I found the following commit which is in v14, but not in v13: ba71918
Thus, the issue seems that we cannot set the required scope in the external storage configuration UI. Strangely, it also does not work when using OpenStack v2 authentication.
Reacted by Volker and Vladimir EliseevNC 14.0.6 still can not integrate with Openstack Object storage (Mitaka + Ceph Jewel)
Nothing is showing in logs (NC and Keystone). Identity endpoint is v3.Hi @teleassist
this is maybe off topic, but you can run into it later: I noticed you use
urn:oid:object prefix. Similar to my case (I haveid:prefix for objects). I have other problem and found that there is problem in Guzzlehttp. Maybe this patch (tuned for you) will help you, please try and give me feedback. The problem I found is that the library handles wrongly:in relative urls.--- nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php 2019-01-01 03:35:39.796954000 +0100 +++ nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php 2019-01-01 03:35:39.796954000 +0100 @@ -142,6 +142,9 @@ private function buildUri($uri, array $config) { + // fix by VP + $uri = str_replace("/urn:oid:", "/urn%3Aoid%3A", $uri); + // for BC we accept null which would otherwise fail in uri_for $uri = Psr7\uri_for($uri === null ? '' : $uri);NC 15, same here!
I also encountered this issue today and got the following log message:
{"reqId":"<redacted>","level":1,"time":"November 21, 2018 21:38:20","remoteAddr":"<redacted>","user":"felix","app":"no app in context","method":"PUT","url":"\/apps\/files_external\/globalstorages\/1","message":"External storage not available: Storage unauthorized. Scope has to be defined for V3 requests","userAgent":"<redacted>","version":"14.0.3.0"}Searching the last part of the message I found the following commit which is in v14, but not in v13: ba71918
Thus, the issue seems that we cannot set the required scope in the external storage configuration UI. Strangely, it also does not work when using OpenStack v2 authentication.
Yes. There is no option to set the scope.
If you change the file like below it should be possible to define a scope. Unfortunately I don't have a working test setup for OpenStack 😞
new DefinitionParameter('url', $l->t('Identity endpoint URL')), new DefinitionParameter('scope', $l->t('Scope'))15 remaining items
@remidebette as your the one who verifyed its working can you make a conclusion did you comment out the
https://github.com/nextcloud/server/blob/master/lib/private/Files/ObjectStore/SwiftFactory.php#L134-L136# if (!isset($this->params['scope'])) { # throw new StorageAuthException('Scope has to be defined for V3 requests'); # } #?
is it still needed? or do you simply supply a fake scope?I can confirm it is still required to comment out these lines because there's no way to set the scope (at least in the external storage GUI).
is it still needed? or do you simply supply a fake scope?
I'm using swift as primary storage and setting scope made it work, no need to edit any nextcloud code.
is it still needed? or do you simply supply a fake scope?
I'm using swift as primary storage and setting scope made it work, no need to edit any nextcloud code.
Hi, can you please provide your config.php, i'm stuck with V2 and can't get V3 to work. Thanks
Edit: I think I got it
' objectstore' => array( 'class' => 'OC\\Files\\ObjectStore\\Swift', 'arguments' => array( 'autocreate' => true, 'user' => [ 'name' => 'MyUserName', 'password' => 'SupaStrongPasswd', 'domain' => [ 'name' => 'Default' ] ], 'scope' => [ 'project' => [ 'name' => 'OS_PROJECT_NAME', 'domain' => [ 'name' => 'Default', ], ], ], 'serviceName' => 'swift', 'region' => 'GRA', 'url' => "https://auth.cloud.ovh.net/v3", 'bucket' => 'nextcloud' ) ),I had to clean files in CLI to get it working
sudo -u www-data php occ files:cleanupReacted by xThrodx and AV Media DesignThanks a lot, I managed to make it work with Keystone v3, the mail documentation is confusing, I plan to fix it today as soon as I have time:
https://docs.nextcloud.com/server/18/admin_manual/configuration_files/primary_storage.html#openstack-swift.Edit: I think I got it
' objectstore' => array( 'class' => 'OC\\Files\\ObjectStore\\Swift', 'arguments' => array( 'autocreate' => true, 'user' => [ 'name' => 'MyUserName', 'password' => 'SupaStrongPasswd', 'domain' => [ 'name' => 'Default' ] ], 'scope' => [ 'project' => [ 'name' => 'OS_PROJECT_NAME', 'domain' => [ 'name' => 'Default', ], ], ], 'serviceName' => 'swift', 'region' => 'GRA', 'url' => "https://auth.cloud.ovh.net/v3", 'bucket' => 'nextcloud' ) ),Yeah, that's it.
I had to clean files in CLI to get it working
sudo -u www-data php occ files:cleanupMine was throwing an error when I left it (late at night), but started working apparently a couple minutes later (as I checked the next day).
Reacted by fxeon and francois1059The cleanup part might indeed not be needed. It happened that I could gain access to my nextcloud page right after I did it. Beforehand it kept throwing me "Storage temporarily not available". As it was late as well I didn't investigate any further.
Please, do not forget that External Storage to OVH is also impacted (the ones configured in database in oc_external_mounts table), and not only Primary Storage.
- added a commit that references this issue
on Mar 4, 2020 - added a commit that references this issue
on Apr 21, 2020 Hello, after upgrade to Nextcloud 19 OVH external storage stopped working as usual. However, when I selected "External storages" in the Settings I noticed that now there is an additional configuration field: "Tenant:". To find the value I connected to my OVH account, went to Public Cloud project and logged into "Horizon" (Openstack management interface). There I went to the Projects tab. I took 16 digit ID that shows as the name of OVH project and copy/pasted it into the Tenant field in Nextcloud. This worked great and now OVH storage works without any modification of configuration files. So basically it seems to be that the issue described in this topic has been fixed.
Steps to reproduce
Expected behaviour
Should be able to access files
Actual behaviour
Complains that the setup is incorrect. This worked fine in 13.0.0. It appears that the new OpenStack v3 integration has broken previous v2 integration.
Server configuration
Operating system: Ubuntu 16.04.5
Web server: Apache2
Database: MySQL 5.7
PHP version: 7.0
Nextcloud version: (see Nextcloud admin page)
Updated from an older Nextcloud/ownCloud or fresh install: Upgrade from NC
Where did you install Nextcloud from: Zip
Signing status:
Signing status
List of activated apps:
App list
Nextcloud configuration:
Config report
Are you using external storage, if yes which one: Swift
Are you using encryption: no
Are you using an external user-backend, if yes which one: no
LDAP configuration (delete this part if not used)
LDAP config
Client configuration
Browser:
Operating system:
Logs
Web server error log
Web server error log
Nextcloud log (data/nextcloud.log)
Nextcloud log
Browser log
Browser log