Invoke-WebRequest does not support UTF-8 (СКУД)

@## Steps to reproduce
Assuming you have a URI that returns json and contains a non-Latin char
My sample is a Confluence wiki page which I am requesting with REST

$splatParameters = @{
              Uri             = "https://powershell.atlassian.net/wiki/rest/api/content/13009245?expand=space,version,body.storage,ancestors&limit=25"
              Method          = "GET"
              Headers         = @{
  "Accept-Charset"                = "utf-8"
  "Accept"                        = "application/json"
  "Content-Type"                  = "application/json; charset=utf-8"
  "Authorization"                 = "Basic abcd"
  }
              UseBasicParsing = $true
              ErrorAction     = 'SilentlyContinue'
          };
  Set-Content -Value (Invoke-WebRequest @splatParameters).Content -Path c:\temp\export.txt -Encoding UTF8 -force

Expected behavior

Invoke-WebRequest is able to write the file with full support of the unicode chars of the source.
In this example: Схема СКУД

Actual behavior

The REST call should return a json like this:

{"id":"13009245","type":"page","status":"current","title":"Схема СКУД","space":{....

instead, the content of c:\temp\export.txt (see example ebove) contains:

{"id":"13009245","type":"page","status":"current","title":"Ð¡Ñ�ÐµÐ¼Ð° Ð¡Ð�Ð£Ð�","space":{.....

Environment data

PS C:\Program Files\PowerShell\6.0.0-rc> $PSVersiontable

Name                           Value
----                           -----
PSVersion                      6.0.0-rc
PSEdition                      Core
GitCommitId                    v6.0.0-rc
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Oliver Lipkau (@lipkau)

Oliver Lipkau (@lipkau) Have you reproduced this issue in 6.0.0-rc? This repository is for PowerShell Core issues. if you have an issue with Windows PowerShell 5.1, please use the UserVoice.

Joel Bennett (@Jaykul)

Yes... I am waiting on Joel Bennett (@Jaykul) for the $PSVersionTable (he tested it for me)

VersionTable updated

Since this issue has some non-latin characters in it and is accessible via an API that returns application/json the following should be an equivalent repro:

$response = Invoke-WebRequest 'https://api.github.com/repos/PowerShell/PowerShell/issues/5528'
Set-Content -Value $response.Content -Path c:\temp\export.txt -Encoding UTF8 -force

When I do that, I see Схема СКУД in the file.

Are you sure that 1) the API you are working with is actually returning UTF8 encoding and 2) the body of that content is actually encoded in utf8?

$response.BaseResponse.Content.Headers.ContentType.CharSet

This seems to be a Confluence API problem.
Closing this issue.

Navigation Menu

Repository navigation

Invoke-WebRequest does not support UTF-8 (СКУД) #5528

Expected behavior

Actual behavior

Environment data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Invoke-WebRequest does not support UTF-8 (СКУД) #5528

Description

Expected behavior

Actual behavior

Environment data

Activity

markekraus commented on Nov 22, 2017

lipkau commented on Nov 22, 2017

lipkau commented on Nov 22, 2017

markekraus commented on Nov 22, 2017

lipkau commented on Nov 22, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions