Repository navigation
Configuring S3 and OpenStack Swift as Primary Storage #60
Description
Activity
You could use a normal object store as primary storage (like Swift, but not S3) https://doc.owncloud.org/server/9.0/admin_manual/configuration_server/config_sample_php_parameters.html#using-object-store-with-owncloud
It would be great to be able to mount Amazon S3 as primary storage !
Reacted by YANO Tetsuro, William Bargent-Ball, Rob Mosher, Andrew Bucklin, Dragan Espenschied, Marius Blüm, cyb0x, Christian Paul, Axel Bock and David OkwiiS3 is still generating lots of overhead connections. owncloud/core#13147
Yes. I'm working on this. I have a working prototype locally. Will try to get this up this week.
Reacted by cyb0x, Marius Blüm, Chris Knutson, Mark Seddon and sumanyamalso a migration mechanism would be really nice then. I am very much looking forward to this feature if it's coming!
@MorrisJobke did this get anywhere? I'm just setting up NextCloud and would really really like this feature!
@MorrisJobke did this get anywhere? I'm just setting up NextCloud and would really really like this feature!
This was taken over by @icewind1991 who did some code in #768
I would also be excited to see this come in the next release.
...maybe we could also introduce a function in the external_files which allows admins to tick a box called "use as primary storage" for "group/user/all" 🤔
7 remaining items
#2315 now open :) . thx!
From what I can tell looking at the merge, this only covered known Amazon S3 endpoints with hard-coded regions. Is there plan to support S3 Compatible (i.e. Ceph) backends as primary storage. This is an OC Enterprise feature keeping me from going to NextCloud at the moment.
Also I should mention the lack of Swift Auth V1 support in OC Enterprise was a big problem, since object storage deployments using only Swift, and not all of OpenStack, use only the V1 Auth and do not have V2 (keystone) implemented.
From what I can tell looking at the merge, this only covered known Amazon S3 endpoints with hard-coded regions. Is there plan to support S3 Compatible (i.e. Ceph) backends as primary storage. This is an OC Enterprise feature keeping me from going to NextCloud at the moment.
This should work. We tested also with different local implementations of the S3 APIs.
Also I should mention the lack of Swift Auth V1 support in OC Enterprise was a big problem, since object storage deployments using only Swift, and not all of OpenStack, use only the V1 Auth and do not have V2 (keystone) implemented.
We plan to also update the underlying library to support new Auth levels. But I don't know what is currently supported. Maybe @icewind1991 knows more here.
Any news about this feature? I would also like to connect NextCloud with SwiftStack or OpenIO via S3 API.
@markseddon any idea if NextCloud + s3fs-fuse is a good idea in production? Never tried it.
- Reacted by Razvan Rosca, Christoph Schaefer and Franco Gilio
@MariusBluem is it production ready, 100+ users?
Feel free to test it in a staging-instance and report us about your experience - I think it is as production-ready as every other part of the Nextcloud server ...
Of course, if you find bugs - please report back!
You may also want to ask in our forum (https://help.nextcloud.com) about the experiences of other administrators - but, to be honest: I think it has not such a large user-base as maybe the calendar-app has.
If you need enterprise support or custom consulting about S3 implementation or Nextcloud in general (especially for larger instances in a critical eviroment) you are welcome to request a quote by the Nextcloud GmbH team over here: https://nextcloud.com/enterprise
Reacted by Franco Gilio@MorrisJobke
Is S3 as primary storage working. https://github.com/nextcloud/files_primary_s3 is not found. what is the correct git url for this S3 implementation.S3 as primary storage is part of the server and has no own repository. You will find instructions how to set this up in our documentation: https://docs.nextcloud.com/server/12/admin_manual/configuration_files/primary_storage.html
Hello, Im wondering if would be possible to make the same thing with Google Drive. Thnx
@MariusBluem That documentation doesn't appear to exist anymore and running a search for 'primary', 'primary storage' etc does not give any useful results :<
Same for me....
The instructions are still available for version 11, I've tested with 13 and you can still mount S3 as primary storage.
https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html
My config file looks similar to Dan_Smith's file from here:
https://help.nextcloud.com/t/external-storage-set-amazon-s3-region/13254/7
'objectstore' => array( 'class' => 'OC\\Files\\ObjectStore\\S3', 'arguments' => array( 'bucket' => 'bucket-name-in-here', 'autocreate' => true, 'key' => 'awskeyinhere', 'secret' => 'awssecretinhere', 'use_ssl' => true, 'region' => 'aws-region-in-here' ), ),how about mount the bucket as /var/www/nextcloud/data ? I am using the oss object storage offered by aliyun in China, and can mount it as a disk by using the ossfs forked from s3fs. And which performance concerns I should pay attention to?
Or better to mount the buckets follow the instruction https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html and as
@jdenly did? But oss is a bit different as Amazon s3.
The instructions are still available for version 11, I've tested with 13 and you can still mount S3 as primary storage.
https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html
My config file looks similar to Dan_Smith's file from here:
https://help.nextcloud.com/t/external-storage-set-amazon-s3-region/13254/7
'objectstore' => array( 'class' => 'OC\\Files\\ObjectStore\\S3', 'arguments' => array( 'bucket' => 'bucket-name-in-here', 'autocreate' => true, 'key' => 'awskeyinhere', 'secret' => 'awssecretinhere', 'use_ssl' => true, 'region' => 'aws-region-in-here' ), ),
Currently it is only possible to mount S3, Google Drive, Dropbox etc. as external storage in a subfolder.
Mounting S3 as primary storage is implanted in the Enterprise Edition. This could be interesting for the community too, so I think we should rewrite it. What do you think?
Reference: https://doc.owncloud.org/server/8.0/admin_manual/enterprise_external_storage/s3_swift_as_primary_object_store_configuration.html