Automatic $? variable (success indicator) is reset to $true when a command is enclosed in parentheses - make $? only reflect command status, not expression status

Not sure if this behavior is by design; if so, it would help to understand the rationale.

Steps to reproduce

Get-Item -EA SilentlyContinue NoSuchItem; $?  # Failure is reflected in $? being $False
(Get-Item -EA SilentlyContinue NoSuchItem); $? #  Using (...) resets $? to $True

Expected behavior

False
False

Actual behavior

False
True

Environment data

PowerShell Core v6.0.0-alpha (v6.0.0-alpha.17) on Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64

I'm seeing the same behavior. I'm not sure what the rationale for this is though, or if it's a bug. It seems like this is a bug to me, but open to ideas.

$? tell us a status of last command. See docs.
$? is $true tell us that "The last command was successful".
In your sample the command is Expression () and the expression was executed successfully.

Ilya (@iSazonov)

Ilya (@iSazonov):

I see. So you're saying that (...) is an expression in its own right? If so, does it ever reflect $False? In other words: is its $? status useful in any way?

Why does the following, which also uses (), yield $False, preserving the status of the enclosed the command?

> (5 + 'not a number'); $?
False

$? returns a Command status not Expression.
The first sample is "command in expresson" and your sample is "command is an expression"

(Invoke-Expression -Command '(5/0); $?' -EA SilentlyContinue); $?

Ilya (@iSazonov)

Ilya (@iSazonov) That's helpful, thanks.

The documentation talks about the "last operation", without defining what operation means in this context.

In practice, as you've demonstrated, that means it's about the last command OR expression.

By command I mean specifically mean something parsed in argument mode (a single function/cmdlet call/external utility call), as opposed to an expression. I'm not sure if there's an established umbrella term for both (perhaps the aforementioned operation? or statement?), but calling the umbrella term "command" too is obviously not a good choice.

Note that even if we assume that "operation" means command or expression, the claim that $? reflects the last operation isn't always true:

Exceptions:

> Get-Item NoSuchItem, / 2>$null | Get-Date; $?
Wednesday, March 15, 2017 5:19:57 PM
False

Even though Get-Date succeeded, $? still reflects the (partial) failure of Get-Item.

Another exception appears to be when the last expression is executed inside a conditional:

do { Get-Item NoSuchItem 2>$null } until ($true); $?
False

Even though true is the last expression evaluated - and was clearly evaluated successfully - $? still reflects the failure of Get-Item.

There is a case where the command's $? status is preserved, at least a command in the syntactical sense: if a command by the specified name cannot be found:

(noSuchCmd); $?
False

Despite embedding in an expression, the inability to find the command is reflected in $? - contrast this with (Get-Item -EA SilentlyContinue NoSuchItem).

Yes, there is a difference between PowerShell not finding the command and an invoked command reporting a non-terminating error, but is it worth treating these two cases differently in terms of their $? behavior?

Conclusions:

Currently, making a command part of an expression invariably loses its $? status - except if the command name isn't recognized at all.
You could make the case that errors in expressions should always result in terminating errors [update: I meant script-terminating errors, such as generated with Throw] that should be handled with try / catch, with $? only reporting command errors, irrespective of whether a command is part of an expression or not.
Does it really make sense to treat 1 / 0 as a non-terminating error, for instance? [update: in reality it is a statement-terminating error, which, however, means that execution still continues by default (with the next statement)]

I believe "Last operation" is exclusively "Command". It is include "Command" in pipeline. No Expression.

do { Get-Item NoSuchItem 2>$null } until (Get-Date); $?
true

One exclusion is for Condition:

// If we're generating code for a condition and the condition contains some command invocation,
// we want to be sure that $? is set to true even if the condition fails, e.g.:
// if (get-command foo -ea SilentlyContinue) { foo }
// $? # never $false here
// Many conditions don't invoke commands though, and in trivial empty loops, setting $? = $true
// does have a measurable impact, so only set $? = $true if the condition might change $? to $false.
// We do this after evaluating the condition so that you could do something like:
// if ((dir file1,file2 -ea SilentlyContinue) -and $?) { <# files both exist, otherwise $? would be $false if 0 or 1 files existed #> }

Ilya (@iSazonov)

Ilya (@iSazonov)

I believe "Last operation" is exclusively "Command".

I wish that were true; it was my initial expectation, reflected in the first post.

However, as you pointed out, if the entire "operation" is an expression (unrelated to a conditional), $? reflects the expression's success, not that of the embedded command:

(Get-Item -EA SilentlyContinue NoSuchItem); $? yields True, with the expression's success eclipsing Get-Item's failure.

Excluding commands in conditionals makes sense, but note that failing expressions are still reflected in $?:

> do { Get-Date } while (1/0); $?
...
False

This is a contrived example, but it leads me back to my point that perhaps expressions themselves should never set $?, only commands (outside conditionals).

And even if we assume that the current behavior is by design,

the rules, as in other cases, are hard to remember.
the rules aren't adequately explained in Get-Help about_Automatic_Variables.

Seems like we may just need better documentation

Michael Klement (@mklement0)

Michael Klement (@mklement0)

However, as you pointed out, if the entire "operation" is an expression

No. You means my:

In your sample the command is Expression () and the expression was executed successfully.

You should understand this as:

The command consists of one expression

And yes, we need update docs.

Steve Lee (@SteveL-MSFT)

Steve Lee (@SteveL-MSFT), Ilya (@iSazonov): I'm glad that we agree that the docs need updating.

For the sake of updating the docs, let's see if we agree what the current behavior is:

The command consists of one expression

Again, using the term "command" in this abstract way is inviting confusion, given the fundamental distinction between a command proper (something parsed in argument mode) and an expression (something parsed in expression mode).

When I use the terms "command" an "expression" below, I'm referring to their specific definitions, and by "operation" I mean the abstraction over the two, excluding multi-command pipelines.

I'm not covering the case where terminating errors are treated as non-terminating ones.
Also note that the behavior described was arrived at by trial and error, not source-code analysis.

Current behavior:

If a command is executed by itself, Boolean variable $? reflects whether it ran successfully; success / failure is determined by (a) if a cmdlet/script/function generated at least one non-terminating error, or (b) in the case of calling an external utility, if its exit code is zero (zero indicating success).
Inside a conditional, command success is by design not reflected in $? (but a failing expression is - see below).
If a command is embedded in an expression - whether via (), $(), or @() - its $? status is lost, because the success of the expression itself is reflected in $?
- Reasons why expressions may fail include: invalid operands (1 / 0, 'foo' -replace 'bar', 'baz', 'ohno' and .NET method calls that throw exceptions ([int]::Parse('foo'))
In a pipeline composed of at least 2 commands, $? reflects whether any of the commands involved reported at least one non-terminating error, not the last one in the pipeline.
All of the above is overruled in the following cases: $? is always $False if the operation involves (a) a nonexistent command or (b) a cmdlet/script/function invoked with an invalid parameter name or (c) a failed expression - in the case of commands, that applies whether or not an expression is involved, and whether or not the command is called in a conditional. [Update: In other words: what matters is that a statement-terminating error occurred.]

If the rules governing the current behavior are somewhat confusing, that's precisely my point:

Expression status overruling command status makes the command status unavailable and complicates things needlessly.
Making expressions never update $? would fix that.
[Update: This paragraph is ultimately about whether statement-terminating errors should be fatal by default - see this thread] Treating failed expressions somewhat like nonterminating errors is problematic in general.
Arguably, a failed expression should always be a terminating error that needs to be handled with try / catch.
Note that failed expressions already are somewhat of a "hybrid" in that while they do not cause termination, you can use try / catch to catch them - something you cannot do with non-terminating command errors.

I'm happy to create a new issue focused on just that.

Bruce Payette (@BrucePay)

CC Bruce Payette (@BrucePay)

Michael Klement (@mklement0)

Michael Klement (@mklement0) Forget "expression". The key here is:

$? returns a Command status not Expression.

Ilya (@iSazonov)

Ilya (@iSazonov):

If we leave the terminology discussion (see below) and to what extent the current docs reflect the intent of the by-design behavior aside, I've tried to point out problematic aspects of the current behavior, and I'd love to hear your thoughts on that:

Aside from the terms that I used, is there something incorrect about how I've described the current behavior?
Is it clear what I've described as problematic about the current behavior (terminology, history, docs aside)?

It seems that there's still confusion over terminology.

Let's first agree that the following two "commands" are different; in fact, their different behavior with respect to $? was the very reason for starting this thread:

(a) Get-Item NoSuchItem - what I've called "command"

(b) (Get-Item NoSuchItem) - what I've called "expression", though, perhaps more accurately, a "value expression", according to Get-Help about_Parsing

From what I understand you've chosen to use "command" as the umbrella term for both forms, something that is:

not reflected in the Get-Help about_Automatic_Variables help topic, which, as discussed, uses the somewhat nebulous term "operation".
at odds with how the term "command" is used in Get-Help about_Command_Syntax, which uses "command" to mean (a) only.

I believe you skip the trick: there are commands consisting of one expression:

Get-Item NoSuchItem - Command
( Get-Item NoSuchItem ) - looks as Expression
( Get-Item NoSuchItem ); - in this context it is Command
if ( Get-Item NoSuchItem ) {} - in this context it is Expression consisting of one Command

So your "current behavior" description should consider it to be extremely accurate.
I also believe that we cannot create correct documentation without formal description language. So we get the exact terms. I believe you are somewhere nearby.
We already have Issue #2254 "Add PowerShell language specification document". Welcome! 😄

Ilya (@iSazonov)

Ilya (@iSazonov):

Thanks for the link.

My final thought re terminology (and I'm happy to leave it at that) is that it's confusing to use the same name ("command") for both something specific ((a) above) and an abstraction from it that encompasses other specific things ((a) and (b) above) - perhaps "statement" is the best abstract term, given that it's already being used in many conceptual topics.

To use a natural language analogy: The same problem has given native English speakers the perennial debate over whether a thumb is a finger, because "finger" can be both something specific (a digit of the hand other than a thumb) and something more abstract (any digit of the hand, including the thumb).

PowerShell parser use "statement" 😄 (You can see this in PowerShell notation.)

Two comments:

(1) $? applies to expressions as well as commands:

Attempted to divide by zero.
At line:1 char:1
+ 1/0; $?
+ ~~~
    + CategoryInfo          : NotSpecified: (:) [], RuntimeException
    + FullyQualifiedErrorId : RuntimeException

False
PS[1] (43) > 1/1; $?
1
True

(2) The best way to think about commands in parenthesis is as follows:

(get-item blargh) ; $?

is "equivalent" to:

write-output (get-item blargh) ; $?

In other words, a command in parenthesis is evaluated, the result is returned and then a second command is used to output those results. Even if the command in parens fails with an error, the parent command succeeds setting $? to true.

Bruce Payette (@BrucePay)

Hmm. I see the rationale there in part, Bruce Payette (@BrucePay)... but this behaviour is always going to be confusing and counterintuitive to users.

Chiefly because in a lot of ways, the following are otherwise perfectly equivalent:

Get-Item blargh
(Get-Item blargh)

Both of them are written to the pipeline in the same way in almost all circumstances. The only case (I can think of) where it differs is if they are part of a pipeline:

(Get-Item blargh) | Select Name, Length
Get-Item blargh | Select Name, Length

And even then, the behavioural change is often not particularly noticeable (though handy to know at times).

That this creates a difference in $? is probably the only really noticeable difference. And given that when using (Command) syntax there is only ever one possible result, I don't think it's a useful difference in any way?

Given also that the current behaviour will complicate working with Rob Holt (@rjmholt)'s chaining operators (#9849) needlessly, I can't help but think it may be time that the behaviour of $? in this case is changed.

Fixed in #11040

Navigation Menu

Repository navigation

Automatic $? variable (success indicator) is reset to $true when a command is enclosed in parentheses - make $? only reflect command status, not expression status #3359

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Automatic $? variable (success indicator) is reset to $true when a command is enclosed in parentheses - make $? only reflect command status, not expression status #3359

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

pcgeek86 commented on Mar 17, 2017

iSazonov commented on Mar 18, 2017

mklement0 commented on Mar 18, 2017

iSazonov commented on Mar 18, 2017

mklement0 commented on Mar 20, 2017

iSazonov commented on Mar 20, 2017

mklement0 commented on Mar 20, 2017

SteveL-MSFT commented on Mar 20, 2017

iSazonov commented on Mar 20, 2017

mklement0 commented on Mar 20, 2017

SteveL-MSFT commented on Mar 20, 2017

iSazonov commented on Mar 20, 2017

mklement0 commented on Mar 20, 2017

iSazonov commented on Mar 21, 2017

mklement0 commented on Mar 21, 2017

iSazonov commented on Mar 21, 2017

BrucePay commented on Mar 12, 2018

vexx32 commented on Sep 6, 2019

rjmholt commented on Jun 16, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions