Repository navigation
Invoke-RestMethod does not strip Authorization Headers #2227
Description
Steps to reproduce
Attempt to download something using Invoke-RestMethod that requires an Authorization header AND generates a 302 redirect to the final location of that resource.
One such example is downloading a build artifact from an AppVeyor project.
E.g from the example here
Invoke-RestMethod -Method Get -Uri "$apiUrl/buildjobs/$jobId/artifacts/$artifactFileName" `
-OutFile $localArtifactPath -Headers @{ "Authorization" = "Bearer $token" }
Expected behavior
The request should succeed like it does on Windows
Actual behavior
The request fails with the following error
Invoke-RestMethod : Response status code does not indicate success: 400 (Authentication information is not given in the correct
format. Check the value of Authorization header.).
At /Users/dave/Desktop/appveyor.ps1:31 char:1
+ Invoke-RestMethod -Method Get -Uri "$apiUrl/buildjobs/$jobId/artifact ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (Method: GET, Re...rShell/6.0.0
}:HttpRequestMessage) [Invoke-RestMethod], HttpRequestException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
Having debugged this with AppVeyor support it seems they send a 302 redirect to a location in Azure. On Windows, I've verified with Fiddler that the Authorization header is stripped which allows this command to succeed. On Mac it would appear the Authorization header is not stripped, causing Azure to generate the error.
Environment data
> $PSVersionTable
Name Value
---- -----
PSVersion 6.0.0-alpha
PSEdition Core
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 3.0.0.0
GitCommitId v6.0.0-alpha.9
CLRVersion
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Activity
ScriptAutomate commentedon Nov 8, 2016
This may be related, otherwise I can open a separate issue.
On PowerShell Core, on both Nano Server and Linux:
[Nano - GA Release - PowerShell Core]
[Linux - PowerShell Core (alpha.11]
Authorization header format should follow:
http://www.ietf.org/rfc/rfc2617.txt
If my Authorization header includes commas, in specific cases, Invoke-WebRequest / Invoke-RestMethod will give a syntax error but do not get this error when on Windows PowerShell.
All I do is remove a single comma after the Credential=VALUE portion, and it passes? Works either way on normal Windows PowerShell.
Note: Example pulled from http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html
Workaround:
It worked when I removed the comma after the Credential=VALUE portion, on both Core and Windows PowerShell, without issue.
SteveL-MSFT commentedon Feb 11, 2017
Derek Ardolf (@ScriptAutomate) you should open that as a separate issue
Dave Tucker (@dave-tucker) I invested this and it seems to be an issue with AppVeyor's REST API (or potentially libcurl). In anycase, if you don't specify the Authorization header, it succeeds. If you specify the Authorization header with curl, it similarly fails. I created a PR to fix their sample appveyor/website#255
SteveL-MSFT commentedon Feb 11, 2017
Looks like the problem is with libcurl which .Net Core depends on. The Auth header is needed by AppVeyor and after auth, they redirect to the Azure Blob Storage which doesn't need it, but because it's there tries to validate and fails. I'll follow-up with libcurl.
SteveL-MSFT commentedon Feb 13, 2017
It doesn't look like libcurl is going to change this behavior. We can handle the 302 redirect in the cmdlets and remove the Auth header by default (make it consistent with Windows) and add a switch to include it on redirect if desired
19 remaining items
dantraMSFT commentedon May 26, 2017
Based on the comments from the committee; I'm going to change the current implementation to default to the behavior and also provide a PreserveAuthorizationOnRedirect to handle edge cases.
The reason for the change is as follows:
On FullCLR, WebRequest is used and, under the hood, the Authorization header is automatically stripped when a redirect occurs. On CoreCLR, HTTPClient is used which doesn't mirror this behavior. To ensure compatibility, I'll update WebRequestPSCmdlet to disable redirects when it detects an authorization header, handle the first redirect by removing the authorization and resubmitting the request to the redirected location with redirection reenabled.
The PreserveAuthorizationOnRedirect will disable this logic for the request for edge cases where the redirected request needs to include the authorization header.
NOTE: There are various discussions floating around about the reverse problem; the header is stripped and callers must handle redirects manually in code. The switch is intended to address this use case when the cmdlet is used.
joeyaiello commentedon May 30, 2017
I think this needs to be merged into the PowerShell master branch before we can close or resolve it.
Add -PreserveAuthorizationHeader on redirect to Invoke-WebRequest and…
Strip authorization header on redirects with web cmdlets (#3885)
Strip authorization header on redirects with web cmdlets (PowerShell#…