Repository navigation
Parameter parsing/passing: an unquoted argument that looks like a named parameter/value pair separated with ":" (colon) is broken in two in positional binding #6292
Description
Note: Unlike the linked issues, the broken behavior described here only affects PowerShell commands, not also native executables.
Steps to reproduce
# Basic function without declared parameters that echoes its arguments via $Args
function Out-Args { $i = 0; $Args | % { 'arg[{0}]: {1}' -f $i++, $_ } }
# Advanced function that binds all arguments via `ValueFromRemainingArguments` and echoes them.
function Out-RemainingArgs { param([parameter(ValueFromRemainingArguments)] $otherArgs) $i = 0; $otherArgs | % { 'arg[{0}]: {1}' -f $i++, $_ } }
# Pass a -<param-nam>:<value> argument that looks like a named parameter, but isn't -
# it is bound anonymously, via $Arg / via `ValueFromRemainingArguments`
Out-Args -foo:bar
'---'
Out-RemainingArgs -foo:barExpected behavior
arg[0]: -foo:bar
---
arg[0]: -foo:bar
Actual behavior
arg[0]: -foo:
arg[1]: bar
---
arg[0]: -foo:
arg[1]: bar
That is, the single $Args / ValueFromRemainingArguments-bound argument was unexpectedly broken in two.
Note that, by contrast, the following cases DO work correctly:
-
When passing tokens such as
-foo:barto external programs - because special effort is already being expended to make this case work (which is needed, because initially all such tokens are parsed as if they were named parameters, even if they later turn out not to be - see @BrucePay's comment below.)- However, when passing such arguments indirectly to external programs, via
$Argsor@Args, the breaking-in-two problem surfaces too - see Parameter parsing/passing: unquoted tokens that look like named arguments with colon as the separator are broken in two when passed indirectly via $Args / @Args #6360
- However, when passing such arguments indirectly to external programs, via
-
Had there been a named parameter
$footo bind to, things would have worked as expected (the token would have been recognized as a parameter-name/value pair: the-foopart would have been used to identify the target parameter, and thebarpart would have become the parameter variable's value).
Environment data
PowerShell Core v6.0.1 on macOS 10.13.3
PowerShell Core v6.0.1 on Ubuntu 16.04.3 LTS
PowerShell Core v6.0.1 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.674 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Activity
BrucePay commentedon Mar 7, 2018
Hi Michael Klement (@mklement0) - this is actually by design. A ':' indicates the end of the parameter name and it doesn't need to be the last character in the string. If it's in the middle of the string, everything after the ':' is treated as the argument to that parameter. Handling parameters this way parallels how a lot of Windows command line tools parse their arguments.
mklement0 commentedon Mar 7, 2018
Bruce Payette (@BrucePay):
Understood, but that applies to named binding: if
-Foo:barbinds to explicitly declared parameter-Foo, the-Fooprefix is solely used for matching and eliminated during binding, so that the parameter variable$Foothen only receivesbar.By contrast, in anonymous binding I would expect a token such as
-Foo:barto be passed through as-is (after potential up-front expansions) - no assumptions should be made about what that token represents, even if it looks like a parameter-name/argument pair.Note that the problem only arises with PowerShell functions, not with external programs: with the latter, commendably,
-foo:baris passed through as-is.I don't know enough about PowerShell's parameter binding to assess whether the distinction I'm asking for presents a conceptual / technical challenge.
lzybkr commentedon Mar 8, 2018
At the parser level, the parameter token ends with
:and the value after that (space or not) is a new token. The:is just used by the parameter binder primarily when binding switch parameters where an argument is unexpected.So from that point of view, this is easily explained and consistent.
There is of course the issue with native commands - there was some effort to not add a space between the arguments when that space did not appear in the original script.
There is a small problem with the current approach though - splatting to a native command will unfortunately add a space:
This outputs:
I suppose this is actually an independent bug, though changing behavior here would address the splatting issue as well (though it's not necessarily a good idea to change both - fixing splatting could be sufficient.)
mklement0 commentedon Mar 11, 2018
Thanks for the background information, Jason Shirk (@lzybkr).
Understood, but my point is that, purely from a conceptual perspective, something that isn't a named argument - even though it may look like one - should be passed through as-is - irrespective of whether it is passed to an external program or to a PowerShell cmdlet/function/script.
The current behavior is inconsistent in that the token is parsed as a named argument - and altered in the process - yet ultimately passed as an unnamed one or, rather, two unnamed ones - and that's the problem.
Of course, a simple workaround is to quote the argument:
Out-Argument '-foo:bar'I've created a separate issue for the splatting bug: #6360
BrucePay commentedon Mar 12, 2018
Michael Klement (@mklement0) To be clear, parameters and arguments are only parsed once when the script is compiled. The command to run is not involved in parsing at all. Command resolution and parameter binding are separate steps done each time a command is executed, potentially long after the parse was completed. So, from the language perspective,
-foo:baris always parsed as parameter-fooand argumentbar. For cmdlets (or script/functions with cmdlet binding) if the command doesn't define a foo parameter, specifying -foo is an error. For native commands, which have no parameter metadata, it's always treated as a value. There is currently a limited "hack" that allows the native command parameter binder to stitch literal arguments like-foo:barback together by looking at the parameter token. See functionBindParameters()in NativeCommandParameterBinder.csmklement0 commentedon Mar 18, 2018
Bruce Payette (@BrucePay): Thanks for this succinct peek behind the scenes and the pointer to the source code - much appreciated.
Based on #6360, as diagnosed by Jason Shirk (@lzybkr), we now know that the limited "hack" needs fixing for
$Args-based parameter passing to external programs.In the interest of a consistent user experience, my plea is to take this opportunity to apply the same "hack" to (directly or splatted) arguments passed to cmdlets/ functions too:
$ArgsValueFromRemainingArgumentsparameterSomething that - even though later, for technical reasons - turns out not to be a parameter, can be dealt with consistently only in one of two ways:
EITHER reject the ultimate non-parameter as unbindable and report an error; tell users to use quoting in such cases.
OR - and that's my plea - apply the limited "hack" consistently to piece the (potentially expanded) token halves back together, irrespective of whether the executable is an external one or a PowerShell cmdlet / function.
If you ultimately don't know what a token such as
-foo:barmeans, it makes no sense to pass it as two arguments - that is potentially more harmful than reporting an error.I've updated the initial post to reflect the current state of our discussion.
[-]Parameter parsing/passing: an unquoted argument that looks like a named parameter/value pair separated with ":" (colon) is broken in two in anonymous binding[/-][+]Parameter parsing/passing: an unquoted argument that looks like a named parameter/value pair separated with ":" (colon) is broken in two in positional binding[/+]microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 16, 2023
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.
microsoft-github-policy-service commentedon Nov 23, 2023
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.