Arguments for external executables aren't correctly escaped

Steps to reproduce

write a C program native.exe which acquires ARGV
Run native.exe "`"a`""

Expected behavior

ARGV[1] == "a"

Actual behavior

ARGV[1] == a

Environment data

Windows 10 x64

Name                           Value
----                           -----
PSVersion                      5.1.14393.0
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.0
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Why do you think that "\"a\"" the expected behavior? My understanding of PowerShell escapes says that the actual behavior is the correct and expected behavior. ""a"" is a pair of quotes surrounding an escaped pair of quotes surrounding an a, so PowerShell interprets the outer unescaped pair as "this is a string argument" and so drops them, then interprets the escaped pair as escaped quotes and so keeps them, leaving you with "a". At no point was a \ added to the string.

The fact that Bash uses \ as an escape character is irrelevant. In PowerShell, the escape character is a backtick. See PowerShell escape characters.

If you want to pass literally "\"a\"", I believe you would use:

> echo `"\`"a\`"`"
"\"a\""

Andy Jordan (@andschwa)

Andy Jordan (@andschwa)
Yes, escapes works fine for internal cmdlets, but things get weird when communicate with native binaries, especially on Windows.
When running native.exe ""a"", the ARGV[1] should be

"a"

(three characters)

instead of

(one character).

Currently to make native.exe correctly receive an ARGV with two quotes and an a character, you have to use this weird call:

native.exe "\`"a\`""

Ah, I see. Re-opening.

Out of a strong curiosity, what happens if you try a build using #1639?

Andy Jordan (@andschwa)

Andy Jordan (@andschwa) The same. You HAVE to double-esacpe to satisify both PowerShell and CommandLineToArgvW. This line:

native.exe "`"a`""

results a StartProcess equalivent to cmd

native.exe ""a""

Belleve (@be5invis)

Belleve (@be5invis) Douglas Thrift (@douglaswth) is this resolved via #2182?

No, We still need to add a backslash before a backtick-escaped double quote? This does not solve the double-escaping problem. (That is, we have to escape a double quote for both PowerShell and CommandLineToArgvW.)

Since ""a"" is equal to '"a"', do you suggest that native.exe '"a"' should result in "\"a\""?

This seems like a feature request that if implemented could break a large number of already existing PowerShell scripts that use the required double escaping, so extreme care would be required with any solution.

sergei (@vors)

sergei (@vors) Yes.
Douglas Thrift (@douglaswth) The double-escaping is really silly: why do we need the “inner” escapes made in the DOS era?

sergei (@vors)

sergei (@vors) Douglas Thrift (@douglaswth)
This is a the C code used to show GetCommandLineW and CommandLineToArgvW results:

#include <stdio.h>
#include <wchar.h>
#include <Windows.h>

int main() {
  LPWSTR cmdline = GetCommandLineW();
  wprintf(L"Command Line : %s\n", cmdline);

  int nArgs;
  LPWSTR *szArglist = CommandLineToArgvW(cmdline, &nArgs);
  if (NULL == szArglist) {
    wprintf(L"CommandLineToArgvW failed\n");
    return 0;
  } else {
    for (int i = 0; i < nArgs; i++) {
      wprintf(L"argv[%d]: %s\n", i, szArglist[i]);
    }
  }
  LocalFree(szArglist);
}

Here is the result

$ ./a "a b"
Command Line : "Z:\playground\ps-cmdline\a.exe" "a b"
argv[0]: Z:\playground\ps-cmdline\a.exe
argv[1]: a b

$ ./a 'a b'
Command Line : "Z:\playground\ps-cmdline\a.exe" "a b"
argv[0]: Z:\playground\ps-cmdline\a.exe
argv[1]: a b

$ ./a 'a"b'
Command Line : "Z:\playground\ps-cmdline\a.exe" a"b
argv[0]: Z:\playground\ps-cmdline\a.exe
argv[1]: ab

$ ./a 'a"b"c'
Command Line : "Z:\playground\ps-cmdline\a.exe" a"b"c
argv[0]: Z:\playground\ps-cmdline\a.exe
argv[1]: abc

$ ./a 'a\"b\"c'
Command Line : "Z:\playground\ps-cmdline\a.exe" a\"b\"c
argv[0]: Z:\playground\ps-cmdline\a.exe
argv[1]: a"b"c

A decision regarding the accommodations on Windows has been made: #15143 (comment).

This appears to be fixed in 7.3:

PS> testexe -echoargs "`"a`""
Arg 0 is <"a">

Steve Lee (@SteveL-MSFT)

This appears to be fixed in 7.3:

Thanks a lot Steve Lee (@SteveL-MSFT). May I know which PR is responsible for the fix?

I can verify double quotes are escaped correctly in v7.3.0-preview.6 when calling a .exe.

> echo $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.0-preview.6
...

> python -c "import sys; print(sys.argv)" '"ab"'
['-c', '"ab"']

However, special characters (", |, &) are still not escaped correctly when invoking a cmd script. Consider a very simply script test.cmd:

echo %*
python -c "import sys; print(sys.argv)" %*

For |, running test.cmd in PowerShell v7.3.0-preview.6 gives:

PS D:\> ./test.cmd "a|b"
'b' is not recognized as an internal or external command,
operable program or batch file.

This differs from the behavior of Command Prompt:

D:\>test.cmd "a|b"

D:\>echo "a|b"
"a|b"

D:\>python -c "import sys; print(sys.argv)" "a|b"
['-c', 'a|b']

This causes problem for Azure CLI when there is | or & in argument value: Azure/azure-cli#20972

For ", running test.cmd in PowerShell v7.3.0-preview.6 gives:

PS D:\> ./test.cmd "`"ab`""

D:\>echo "ab"
"ab"

D:\>python -c "import sys; print(sys.argv)" "ab"
['-c', 'ab']

This differs from the behavior of Command Prompt:

D:\>test.cmd "\"ab\""

D:\>echo "\"ab\""
"\"ab\""

D:\>python -c "import sys; print(sys.argv)" "\"ab\""
['-c', '"ab"']

Jiashuo Li (@jiasli)

As noted by Jiashuo Li (@jiasli), this issue is not actually fixed.

The PowerShell behavior here is the same as bash

test.sh:

#!/bin/bash

echo $@

bburns@DESKTOP-VHJJ57D:~$ ./test.sh "a|b"
a|b
bburns@DESKTOP-VHJJ57D:~$ ./test.sh "a&b"
a&b
bburns@DESKTOP-VHJJ57D:~$

In my opinion, the bash/powershell behavior makes more sense than the cmd behavior. In the above example, the quotes are intended for the shell interpreter, they are not part of the user's input. The shell's job is to pass each value as a String to the command that it calls. It is the executable/command scripts responsibility to re-escape as necessary before shelling out again.

Furthermore this seems to be a specific behavior related to cmd.exe calling command scripts.

import sys
print(sys.argv)

C:\Users\bburns>py test.py 'a|b'
'b'' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\bburns>py test.py 'a&b'
['test.py', "'a"]
'b'' is not recognized as an internal or external command,
operable program or batch file.

Jiashuo Li (@jiasli)

As noted by Jiashuo Li (@jiasli), this issue is not actually fixed.

This issue is about passing arguments to native executables that support passing arguments. The executable cmd.exe is not one of them, it uses the whole command line instead. The only reliable way to call cmd.exe is to print the expected command line out into a temporary command script and run that script without any arguments.

Fix npm "Unknown command: pm" in build-play-aab on Windows PowerShell…

feat!: add the print statement and static modifier

Acik temada terminal acilirken 'Invalid JSON provided to --settings' …

Navigation Menu

Repository navigation

Arguments for external executables aren't correctly escaped #1995

Steps to reproduce

Expected behavior

Actual behavior

Environment data

218 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Arguments for external executables aren't correctly escaped #1995

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

andyleejordan commented on Aug 31, 2016

be5invis commented on Aug 31, 2016

be5invis commented on Aug 31, 2016

andyleejordan commented on Aug 31, 2016

andyleejordan commented on Aug 31, 2016

be5invis commented on Aug 31, 2016

andyleejordan commented on Sep 19, 2016

be5invis commented on Sep 20, 2016

vors commented on Sep 20, 2016

douglaswth commented on Sep 20, 2016

be5invis commented on Sep 20, 2016

be5invis commented on Sep 20, 2016

be5invis commented on Sep 20, 2016

218 remaining items

mklement0 commented on May 6, 2021

SteveL-MSFT commented on Aug 11, 2022

jiasli commented on Aug 11, 2022

jiasli commented on Aug 11, 2022

rivy commented on Aug 11, 2022

brendandburns commented on Aug 11, 2022

brendandburns commented on Aug 11, 2022

yecril71pl commented on Sep 16, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions