Suggestion: implement null-coalescing, null-conditional access (null-soaking), null-conditional assignment

Bruce Payette (@BrucePay)

Null-coalescing and null-conditonal access (null-soaking) would be handy additions to the language.

Update: Bruce Payette (@BrucePay) additionally suggests null-conditional assignment, with ?= - see comment below.

For instance, instead of writing:

if ($null -ne $varThatMayBeNull) { $varThatMayBeNull } else { $fallbackValue }
#
if ($null -ne $varThatMayBeNull) { $varThatMayBeNull.Name } else { $null }

one might be able to write:

$varThatMayBeNull ?? $fallbackValue  # null-coalescing
# 
$varThatMayBeNull?.Name   # null-conditional access, for Set-StrictMode -Version 2+
$varThatMayBeNull?[42]  # ditto, with indexing

Re null-conditional access: With Set-StrictMode being OFF (the default), you can just use $varThatMayBeNull.Name - no special syntax needed; however, if Set-StrictMode -Version 2 or higher is in effect, $varThatMayBeNull.Name would break and that's where the null-conditional operator (?.) is helpful, to signal the explicit intent to ignore the $null in a concise manner.

Open question:

$varThatMayBeNull?[42] handles the case where the variable is $null, but if it isn't, an array element with the specified index must exist.

It would therefore also be helpful to make indexing null-conditional - something that C# does not support, incidentally (you have to use .ElementAtOrDefault()).

The two basic choices are:

Come up with additional syntax that explicitly signal the intent to ignore a non-existent index:
- The question is what syntax to choose for this, given that ?[...] and [...]? are not an option due to ambiguity.
- Perhaps [...?], but that seems awkward .
Rely on the existing behavior with respect to accessing non-existent indices, as implied by the Set-StrictMode setting - see table below.

Related: implement ternary conditionals

Michael Klement (@mklement0)

Michael Klement (@mklement0) It seems the second sample works by design without "?".
Maybe change to name()?

Ilya (@iSazonov)

Ilya (@iSazonov): Good point, but it only works without the ? unless Set-StrictMode -Version 2 or higher is not in effect - I've updated the initial post to make that clear.

I think syntactic sugar is something powershell could use that other languages enjoy. +1 on the proposal.

Consider the following samples from other languages:

a="${b:-$c}"
a = b || c;
a = b or c
a := b ? c
a = b ? b : c;

even

a = b if b else c

is better than

if (b) { a = b } else { a = c }

and often times you need to clarify with the additionally verbose

a=(if (b -ne $null) { b } else { c })

Makes one feel all dirty and bashed.

Found myself doing this today as work around.

$word = ($null, "two", "three").Where({$_ -ne $null}, "First")

I use a similar pattern:

$word = ($null, "two", "three" -ne $null)[0]

Oliver Smith (@kfsone)

Oliver Smith (@kfsone) There is a small error in your example $a=(if ($b -ne $null) { $b } else { $c }). It should be $a=$(if ($b -ne $null) { $b } else { $c }) however the only version of PowerShell where $( ) was required was version 1. From v2 on, you can simply do:

$a = if ($b) { $b } else { $c }

Brian G. Shacklett (@bgshacklett)

Brian G. Shacklett (@bgshacklett) Don't you mean $word=($null,'two')[$null -ne 'three']?

It's a bit unfortunate this has gone from 6.1 to 6.2 to "Future".

Yeah, I'd be a little concerned about tokenizing overhead there. Might be worth implementing both possibilities and doing some benchmarks on a script that uses similar variables names reasonably heavily.

My preference is definitely option 1... a one-time breaking change is, to me at least, way more preferable to having to deal with the inconsistencies in how that can be parsed way into the future.

If so, I think we may have a lot of trouble handling the tokenizing / parsing logic without a lot of backtracking. This might lead to performance degradations when variables end with ?.

I share this concern about that possibility. Having ? as a sometimes-token-separator feels like a jagged line in the language to me.

If so, I think we may have a lot of trouble handling the tokenizing / parsing logic without a lot of backtracking. This might lead to performance degradations when variables end with ?.

It depends on how you implement it. As long as you take a lookahead approach rather than a tokenize and then back-up approach, you should be fine. You can look ahead at what characters are next when you encounter a ? in a variable name, and make a decision on how you want to "wrap up" the variable token based on what's next. That isn't a lot of trouble and shouldn't require backtracking or result in noticable performance degradations.

Dongbo Wang (@daxian-dbw)

@PowerShell/powershell-committee reviewed this one today, we have a couple thoughts:

No matter what we do, we're going to do some analysis of our corpus of scripts to see how often folks use ? in variable names
Some of us have a hypothesis (that others would like to validate) that the users who are using ? in their variable names may be less advanced users (as we agree in this room we'd stay away from it because of the potential problems that could arise). On the other hand, anyone using the functionality described here will be able to understand a slightly more complicated syntax (like ${foo}?.bar). Therefore, we prefer option 3 because it avoids breaking changes on these less experienced users. (But again, we will validate it per the first bullet.)
Dongbo Wang (@daxian-dbw) raised a good point about making it harder to find all variable references when scripters are mixing usage of $foo and ${foo}. However, we agreed that this can be fixed in tooling (like the VS Code PowerShell extension), and users like James Truher (@JamesWTruher) who use editors like Vim can easily match on both with their search syntax.

harder to find all variable references when scripters are mixing usage of $foo and ${foo}

That depends on whether the AST differentiates the variable name based on whether it saw the braces. I suspect that any tool that uses the PowerShell AST will have no trouble with braces here.

But for regex, it certainly means you have to be more aware: varname -> \{?varname\}?

Rob Holt (@rjmholt)

Rob Holt (@rjmholt) I did an analysis over here. A quick breakdown: Out of nearly 400,000 scripts with 22,000,000+ variable names (not unique), there were only ~70 unique variables that ended with ?.

Thanks, @mburszley - to me that makes it a Bucket 3: Unlikely Grey Area change, and burdening user with the need for {...} is highly unfortunate, paralleling the unfortunate need for $(...) around exit / return / throw statements in the context of && and ||.

To borrow the reasoning from that issue:

If we force the use of {...} around variable names just so you can use ?.:

New users will not expect the need for {...} and won't necessarily know that it is what is required when the following fails (possibly undetected, unless Set-StrictMode -Version 2 or higher is in effect): $o = [pscustomobject] @{ one = 1 }; $o?.one
Even once users do know about the need for {...}:
- They will forget to use it on occasion, because of the counter-intuitive need for it.
- When they do remember, this seemingly artificial requirement will be an ongoing source of frustration, especially since {...} is hard to type.

Rob Holt (@rjmholt)

Rob Holt (@rjmholt), quoting you from #10967 (comment):

not breaking the way we parse existing valid syntax was the right way to go. Again, a change there isn't just a case of us making a decision to support a small number of wild programs out there — we've made numerous breaking changes in cases where we thought the pros outweighed the cons (not that I personally agree with all of them, or that those do or don't justify others). The issue is that changing some aspect of the tokenizer or parser means that two PowerShell versions will no longer generate the same AST for some scripts, so two PowerShells will "see" the same script differently. That means we can't even read the syntax the same way, so there's no PSScriptAnalyzer rule you can write to pick up possible issues around it; PSScriptAnalyzer will see a different syntax from one PowerShell version to the next.

Unlike C#, PowerShell can't pre-compile a different syntax to a compatible format for later execution, meaning that a syntactic change is tethered to the runtime. Once PowerShell makes a syntactic break, we reduce the number of scripts that can work against different versions, meaning users are forced to write two scripts, which is a serious issue for a shell and a scripting language. PowerShell is supposed to be dynamic enough to bend past all the differences with logic inside one script, but the syntax is the one non-negotiably static thing we have. And making a syntax change where both before and after are valid is especially pernicious, since there's no simple way to detect it, there's no warning for it and even after executing it in both environments, users might not know that a different behaviour occurred.

In general, I can appreciate that such changes can be very problematic and should only be made if the pros outweigh the cons, an important factor of which is how much existing code breaks.

we reduce the number of scripts that can work against different versions
meaning users are forced to write two scripts

We're talking about a new syntactic feature here (?.), so by definition scripts that use it cannot (meaningfully) run on older versions - unless you specifically add conditionals that provide legacy-version-compatible code paths, but that seems hardly worth it - you would then just stick with the legacy features.

Yes, the interpretation of $var?.foo in old scripts that used $var? as a variable name would break, but:

? should never have been allowed as part of an identifier without enclosing it in {...}.
Since being able to do so is unexpected and probably even unknown to many, such variable names are exceedingly rare in the wild, speaking from personal experience, but @mburszley's analysis provides more tangible evidence.
- Even Ruby only allows ? (and !) at the end of identifiers, and there only of method identifiers, and I suspect that most Ruby users are aware that they should not assume that other languages support the same thing.

So, pragmatically speaking:

The vast majority of existing code will not be affected - a token such as $var?.foo will simply not be encountered.
If you write $var?.foo with the new semantics, then yes, running that on older versions could result in different behavior (rather than breaking in an obvious manner), depending on what strict mode is in effect - but you should always enforce the minimum version required to run your code as intended anyway (#requires -Version, module-manifest keys).

All in all, to me this a clear case of a bucket 3 change: a technically breaking change that breaks very little existing code while offering real benefits (or, conversely, avoiding perennial headaches due to unexpected behavior).

Michael Klement (@mklement0)

Michael Klement (@mklement0) we're keeping the Null-Conditional access experimental for PS7.0. Perhaps you can open a new issue to continue this discussion for 7.1?

Steve Lee (@SteveL-MSFT)

Sounds good, Steve Lee (@SteveL-MSFT) - please see #11379.

I encourage everyone here to again show support (or, gasp, non-support) there.

Navigation Menu

Repository navigation

Suggestion: implement null-coalescing, null-conditional access (null-soaking), null-conditional assignment #3240

56 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Suggestion: implement null-coalescing, null-conditional access (null-soaking), null-conditional assignment #3240

Description

Activity

iSazonov commented on Oct 8, 2017

mklement0 commented on Oct 8, 2017

morsley commented on Jul 24, 2018

kfsone commented on Jul 25, 2018

thezim commented on Aug 17, 2018

bgshacklett commented on Aug 18, 2018

BrucePay commented on Aug 19, 2018

morsley commented on Aug 21, 2018

56 remaining items

vexx32 commented on Aug 23, 2019

rjmholt commented on Aug 23, 2019

KirkMunro commented on Aug 24, 2019

joeyaiello commented on Aug 26, 2019

rjmholt commented on Aug 29, 2019

rjmholt commented on Aug 29, 2019

morsley commented on Aug 29, 2019

mklement0 commented on Nov 24, 2019

mklement0 commented on Dec 14, 2019

SteveL-MSFT commented on Dec 16, 2019

mklement0 commented on Dec 17, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions