SharePoint support by blizzz · Pull Request #4106 · nextcloud/server

blizzz · 2017-03-27T23:17:58Z

Contributes to #2672

Add a SharePoint Backend to files_external

configuration:

requirements

NTLM Auth enabled on SP. We strictly require NTLM Auth. Basic did not work for me, I assume because of the SP/IIS configuration. Apparently NTLM is > then Basic anyhow.

environment

tested and developed against SharePoint 2016, but
SharePoint 2013 should work, however this is not proven
removes queries on the go so they don't get stuck on exceptions vgrem/phpSPO#63 is applied to phpSPO lib

features

dir listings, trying to identify and hide hidden system directories (e.g. "Forms").
creating dirs
down- and upload of files (via streams, exception: new files)
renames
removing files and folders (using recycle method i.e. putting it into the wastebin)
determine CRUD permissions

mention-bot · 2017-03-27T23:18:00Z

@blizzz, thanks for your PR! By analyzing the history of the files in this pull request, we identified @icewind1991, @Xenopathic and @LukasReschke to be potential reviewers.

icewind1991 · 2017-03-28T09:52:30Z

Any chance we can keep the dragons contained in their own app?

blizzz · 2017-03-28T10:21:11Z

@icewind1991 i caught one dragon for you: on mkdir('foo/bar') an opendir against 'foo' is performed. why?

So, the scanner kicks in (external storage), but it is necessary when triggered by httpMkcol()? The direct trigger is getNodeForPath() from our ObjectTree Connector. At this point however there is no information about the operation. Hm.

codecov-io · 2017-03-28T13:13:48Z

Codecov Report

Merging #4106 into master will decrease coverage by 0.44%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4106      +/-   ##
============================================
- Coverage      54.2%   53.76%   -0.45%     
- Complexity    21597    21838     +241     
============================================
  Files          1327     1277      -50     
  Lines         82538    76276    -6262     
  Branches       1305        0    -1305     
============================================
- Hits          44739    41007    -3732     
+ Misses        37799    35269    -2530

Impacted Files	Coverage Δ	Complexity Δ
apps/dav/lib/DAV/Sharing/Backend.php	`71.08% <0%> (-16.87%)`	`17% <0%> (ø)`
apps/sharebymail/lib/Settings/SettingsManager.php	`25% <0%> (-15%)`	`3% <0%> (+1%)`
apps/user_ldap/appinfo/routes.php	`66.66% <0%> (-10.61%)`	`0% <0%> (ø)`
apps/sharebymail/lib/ShareByMailProvider.php	`60.07% <0%> (-8.68%)`	`87% <0%> (+22%)`
apps/comments/lib/EventHandler.php	`79.16% <0%> (-8.34%)`	`7% <0%> (ø)`
settings/Controller/CheckSetupController.php	`70.52% <0%> (-8.1%)`	`56% <0%> (ø)`
apps/dav/lib/CalDAV/Calendar.php	`73.68% <0%> (-3.42%)`	`52% <0%> (+3%)`
apps/user_ldap/lib/User/User.php	`80.53% <0%> (-2.15%)`	`123% <0%> (+32%)`
apps/dav/lib/CardDAV/AddressBook.php	`64.63% <0%> (-2.04%)`	`27% <0%> (+3%)`
apps/user_ldap/lib/User_LDAP.php	`74.35% <0%> (-1.54%)`	`72% <0%> (+3%)`
... and 186 more

Signed-off-by: Arthur Schiwon <[email protected]>

* use NTLM Login only for now * fix fetching Document Library * fix retrieve root folder info Signed-off-by: Arthur Schiwon <[email protected]>

Signed-off-by: Arthur Schiwon <[email protected]>

also simplifies and improves internal fetchFileOrFolder method Signed-off-by: Arthur Schiwon <[email protected]>

* seperates direct backend code and API handling * an awful lot easier to unit test Signed-off-by: Arthur Schiwon <[email protected]>

it appears awefully slow, mostly because of yet missing caching, as other storage backends deal with that. To be added later. Signed-off-by: Arthur Schiwon <[email protected]>

Signed-off-by: Arthur Schiwon <[email protected]>

still takes too long though Signed-off-by: Arthur Schiwon <[email protected]>

Signed-off-by: Arthur Schiwon <[email protected]>

depending on whether a . is present in the path, with fallback. Signed-off-by: Arthur Schiwon <[email protected]>

* look for hidden info only when really necessary * also on for folders * don't retrieve folder contents not with fetchFolder, fetchFolderContents only Signed-off-by: Arthur Schiwon <[email protected]>

Signed-off-by: Arthur Schiwon <[email protected]>

saves re-requesting any of its childeren from SP Signed-off-by: Arthur Schiwon <[email protected]>

includes performance optimization for this and mkdir Signed-off-by: Arthur Schiwon <[email protected]>

Signed-off-by: Arthur Schiwon <[email protected]>

deprecates nextcloud/server#4106 Signed-off-by: Arthur Schiwon <[email protected]>

blizzz · 2017-04-26T15:04:05Z

After some off-github discussions, this is deprecated in favor of a separate app → https://github.com/nextcloud/sharepoint

blizzz added 2. developing Work in progress feature: external storage labels Mar 27, 2017

blizzz added this to the Nextcloud 12.0 milestone Mar 27, 2017

nickvergessen mentioned this pull request Mar 27, 2017

Nextcloud 12 overview #2672

Closed

37 of 59 tasks

blizzz added the enhancement label Mar 28, 2017

blizzz force-pushed the sharepoint-support branch from a73b031 to 1036364 Compare March 31, 2017 10:12

blizzz added 20 commits April 4, 2017 12:25

add phpSPO lib

e7f0fe6

Signed-off-by: Arthur Schiwon <[email protected]>

initial Backend and Storage class

943d990

Signed-off-by: Arthur Schiwon <[email protected]>

WIP changes

0729e05

Signed-off-by: Arthur Schiwon <[email protected]>

DI and init methods, we go for basic auth until we here anything else

04472b1

Signed-off-by: Arthur Schiwon <[email protected]>

work on stat

2a28248

Signed-off-by: Arthur Schiwon <[email protected]>

work on stat and tests

2c181b4

Signed-off-by: Arthur Schiwon <[email protected]>

complete stat(), implement filetype()

b0d8a7a

Signed-off-by: Arthur Schiwon <[email protected]>

implement fileExists() and unlink()

a44f37b

Signed-off-by: Arthur Schiwon <[email protected]>

update php-spo

d895e71

Signed-off-by: Arthur Schiwon <[email protected]>

first testing and fixes

9be6573

* use NTLM Login only for now * fix fetching Document Library * fix retrieve root folder info Signed-off-by: Arthur Schiwon <[email protected]>

polish stat()

2991f39

Signed-off-by: Arthur Schiwon <[email protected]>

rework file_type() and file_exists()

61298bf

also simplifies and improves internal fetchFileOrFolder method Signed-off-by: Arthur Schiwon <[email protected]>

Split SharePoint Backend in two classes

89d48f8

* seperates direct backend code and API handling * an awful lot easier to unit test Signed-off-by: Arthur Schiwon <[email protected]>

implement opendir()

ab1a54f

it appears awefully slow, mostly because of yet missing caching, as other storage backends deal with that. To be added later. Signed-off-by: Arthur Schiwon <[email protected]>

fix interpreting mtime from SP

c1952c0

Signed-off-by: Arthur Schiwon <[email protected]>

performance tuning part 1: add cache, saves 80% time

1f25415

still takes too long though Signed-off-by: Arthur Schiwon <[email protected]>

properly add php-spo

e072312

Signed-off-by: Arthur Schiwon <[email protected]>

perf tuning pt 2. recognize files and folders when blindly fetching them

5228c95

depending on whether a . is present in the path, with fallback. Signed-off-by: Arthur Schiwon <[email protected]>

Perf optimization part 3: further lookup reduction

2d50097

* look for hidden info only when really necessary * also on for folders * don't retrieve folder contents not with fetchFolder, fetchFolderContents only Signed-off-by: Arthur Schiwon <[email protected]>

implement mkdir

0e9681a

Signed-off-by: Arthur Schiwon <[email protected]>

blizzz added 5 commits April 4, 2017 12:25

cache file and folder instances fetched by fetchFolderContents

aeffd16

saves re-requesting any of its childeren from SP Signed-off-by: Arthur Schiwon <[email protected]>

implement rmdir()

620b459

includes performance optimization for this and mkdir Signed-off-by: Arthur Schiwon <[email protected]>

implement read part of fopen

e2c03bb

Signed-off-by: Arthur Schiwon <[email protected]>

implemented fopen

56a2d83

Signed-off-by: Arthur Schiwon <[email protected]>

fixed and improved delete and rename methods

9b79ddb

Signed-off-by: Arthur Schiwon <[email protected]>

blizzz force-pushed the sharepoint-support branch from 1036364 to 9b79ddb Compare April 19, 2017 22:16

blizzz added 6 commits April 20, 2017 13:16

remove resetClientContext() the troublemaker will be fixed in the lib

1ac9a49

some cleanup

72225ab

Signed-off-by: Arthur Schiwon <[email protected]>

get CRUD permissions

007ca14

Signed-off-by: Arthur Schiwon <[email protected]>

add missing PHP Doc

ab29f87

Signed-off-by: Arthur Schiwon <[email protected]>

adjust and extend unit tests

b217488

Signed-off-by: Arthur Schiwon <[email protected]>

Update phpSPO and apply vgrem/phpSPO#63

f022d44

Signed-off-by: Arthur Schiwon <[email protected]>

blizzz changed the title ~~[WIP] SharePoint support~~ SharePoint support Apr 24, 2017

blizzz added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Apr 24, 2017

blizzz requested review from MorrisJobke, icewind1991, nickvergessen, oparoz and rullzer April 24, 2017 23:24

syntax fixes in upstream lib for php < 7.1

90d3656

Signed-off-by: Arthur Schiwon <[email protected]>

blizzz added a commit to nextcloud/sharepoint that referenced this pull request Apr 26, 2017

Make it a separate app

1625c96

deprecates nextcloud/server#4106 Signed-off-by: Arthur Schiwon <[email protected]>

blizzz closed this Apr 26, 2017

MorrisJobke deleted the sharepoint-support branch April 26, 2017 16:12

Navigation Menu

Repository navigation

SharePoint support - #4106

SharePoint support#4106
blizzz wants to merge 32 commits into
masterfrom
sharepoint-support

blizzz commented Mar 27, 2017 •

edited

Loading

Uh oh!

mention-bot commented Mar 27, 2017

Uh oh!

icewind1991 commented Mar 28, 2017

Uh oh!

blizzz commented Mar 28, 2017 •

edited

Loading

Uh oh!

codecov-io commented Mar 28, 2017 •

edited by codecov Bot

Loading

Uh oh!

blizzz commented Apr 26, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

blizzz commented Mar 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

configuration:

requirements

environment

features

Uh oh!

mention-bot commented Mar 27, 2017

Uh oh!

icewind1991 commented Mar 28, 2017

Uh oh!

blizzz commented Mar 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Mar 28, 2017 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blizzz commented Apr 26, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

blizzz commented Mar 27, 2017 •

edited

Loading

blizzz commented Mar 28, 2017 •

edited

Loading

codecov-io commented Mar 28, 2017 •

edited by codecov Bot

Loading