Should we officially "deprecate" Invoke-Expression?

Summary of the new feature/enhancement

Opening issue to get feedback per our shiproom conversation.

We have publicly stated that Invoke-Expression should be avoided. There are too many inherent security risks built into it. We continue to get questions in the Docs repo about how and why it should be used, and asks for useful examples.

We have the ability, in code, to set an attribute on the cmdlet that marks it as deprecated. Should this change be made? Are there compelling cases for the use of this cmdlet?

Sean Wheeler (@sdwheeler)

In the linked issue Sean Wheeler (@sdwheeler) states:

Invoke-Expression is maintained for historical purposes but is effectively deprecated

From Rob Holt (@rjmholt)'s response ibidem:

My personal feeling (worth also talking to Steve Lee (@SteveL-MSFT), Joey Aiello (@joeyaiello), James Truher (@JamesWTruher) and Bruce Payette (@BrucePay)) is that it’s not so much historical as an important part of an interpreted language. It’s dangerous, and shouldn’t be used in production, but all scripting languages have an eval() equivalent.

Also of interest: his comment with specific invocation scenarios, MicrosoftDocs/PowerShell-Docs#4897 (comment), where he concludes:

I think the uses are quite obscure and Invoke-Expression needs no encouragement. We should just warn against it..

As someone who does use Invoke-Expression for good reason, in a small number of scripts, I'm damned if I can see why you want to take away this functionality. It exists in Powershell and exists in some form in every other scripting language. There are plenty of compatibility and performance issues with PS core that need to be addressed before you start breaking my existing scripts.

lukeb1961

I feel deprecating like this is not all that helpful. There are, as lukeb1961 and others point out, useful reasons for using IEX. Not many, but some are certainly 'valid'.

I feel that to document the issue and warn against it is probably enough.

I do not see the use in removing this, when I use it in so many scripts, and have been taught how to use this effectively in so many scripts for API calls too. Please don't remove it.

To be clear, we are not talking about removing Invoke-Expression. We are talking about marking it as "deprecated" and more emphatically warning against its use.

If you have compelling use cases, please share them so we can see WHY it is needed.

Sean Wheeler (@sdwheeler)

I'm damned if I can see why you want to take away this functionality

Nobody is talking about removing Invoke-Expression. It’s an important primitive in PowerShell (as in any scripting language).

Here and in a few other places there has been some confusion about the meaning of the term deprecate. It simply means to express disapproval of, to disrecommend. Obsolescence is one reason to do so, but another is unnecessary danger. Invoke-Expression lies in the second category; it’s not going anywhere but it’s dangerous to use non-interactively, both from a security standpoint and an efficiency one (the whole parser/compiler toolchain has to be run again at execution time).

What we mean, on a technical level, by deprecation in this case is still not clear to me (just in documentation, or with code attributes as well?), but you’ll always be able to use Invoke-Expression (even though you shouldn’t).

As someone who does use Invoke-Expression for good reason, in a small number of scripts

Would you be able to share those scenarios/usages or adaptations of them? I think Sean Wheeler (@sdwheeler) in particular is looking for examples of iex usage.

Rob Holt (@rjmholt)

Rob Holt (@rjmholt) Thanks. Well said. The purpose of this issue was to discuss whether we should add the "deprecated" attribute to the code to enhance the warning about use of the cmdlet. This is separate from the question of how should this be documented.

I am also looking for good example for how and why you are using IEX instead of the the &-operator. If someone can provide a useful example then we can add that to the documentation. Otherwise, I will just update the documentation with a more direct recommendation for not using the cmdlet.

Rob Holt (@rjmholt)

Here and in a few other places there has been some confusion about the meaning of the term deprecate. It simply means to express disapproval of, to disrecommend.

Yeah literally every time the term is used. Y'all may want to adopt a template for using the term in a public space, something like:

IMPORTANT NOTE: This is not about removing ___. (Link to doc on what deprecation means)

The actual post content.

Actually I don't know if anyone would read that. Maybe a different term with less connotation needs to be used instead.

Rob Holt (@rjmholt) Thanks. Well said. The purpose of this issue was to discuss whether we should add the "deprecated" attribute to the code to enhance the warning about use of the cmdlet. This is separate from the question of how should this be documented.

Do you mean ObsoleteAttribute? If so, the runtime warnings would get really annoying. A PSSA rule is where this would need to live imo.

Sean Wheeler (@sdwheeler)

whether we should add the "deprecated" attribute to the code to enhance the warning about use of the cmdlet. This is separate from the question of how should this be documented.

Thanks for that clarification, Sean Wheeler (@sdwheeler); the distinction wasn't obvious to me, and it seems that these aspects have already been conflated here, which is just as well, because I think discussing them in tandem makes sense.

As for the attribute: I agree with Patrick Meinecke (@SeeminglyScience) that a PSSA rule is the right place to warn programmatically, if desired; certainly not at runtime, as that amounts to a quasi-breaking change.

As for documentation and use cases:

In the Invoke-Expression help topic, I think a general warning is sufficient, as Rob Holt (@rjmholt) proposes, along the following lines (adapted from the original MicrosoftDocs/PowerShell-Docs#2679 issue):

As a matter of habit, avoid use of Invoke-Expression, because there are usually more secure and robust alternatives.

If you do find that Invoke-Expression is your only choice, carefully consider the security implications: if a string from an (untrusted) outside source (e.g., user input) is passed directly to Invoke-Expression, arbitrary commands may be executed.

I think that is sufficient for this particular help topic, but separately, perhaps in a new about_Invoke-Expression conceptual help topic, we could focus on when not to use Invoke-Expression.

Specifically, we could take widespread current and historical (mis)uses of Invoke-Expression and show superior, safer alternatives.

If we do end up agreeing on legitimate public uses worth advertising (other than during PowerShell tests, as described by Rob Holt (@rjmholt) in MicrosoftDocs/PowerShell-Docs#4897 (comment)), we can add them there as well.

Even in the perhaps most seductive use case - direct execution of script text downloaded from the web (e.g, iwr https://aka.ms/install-powershell.ps1 | iex) - Invoke-Expression is ultimately ill-advised, and enhancing Invoke-Command to operate on Invoke-WebRequest output should be favored - see #5909 (comment)

In fact PSSA has had a rule to disrecommend Invoke-Expression for a few years now

certainly not at runtime, as that amounts to a quasi-breaking change

While I agree that adding an ObsoleteAttribute would be annoying and probably not what we want here, I strongly disagree that it would be a breaking change.

If adding ObsoleteAttribute to anything is a breaking change, then that attribute is pointless. The whole idea of attributes is that they behave as additive metadata.

Anyway, that’s a tangent.

Specifically, we could take widespread current and historical (mis)uses of Invoke-Expression and show superior, safer alternatives.

Absolutely agreed.

I’d like to build a library of Invoke-Expression use-cases and document alternatives.

Unfortunately in the three issues we’ve been discussing this, we haven’t managed to record any of those uses. Hopefully we can collect some here so we can write or enhance documentation around them.

aescolastico

aescolastico: The & operator does not evaluate PowerShell source code stored in a string the way that Invoke-Expression does.

It only accepts a command name or path as its operand (or, alternatively a script block), so for running external programs the arguments must still be passed separately.

In fact, the only reason & is ever needed when calling external programs is a syntactic one: So that PowerShell knows you're executing a command, & must be used if the executable file path is specified in quotes and/or contains variable references / subexpressions (& "C:\Program Files\Acme\foo.exe" bar or & $HOME\bin\foo.exe bar); otherwise, just invoke the executable directly (C:\Users\jdoe\bin\foo.exe bar).

If you need to construct arguments programmatically, it is much cleaner to create them as elements of an array first, which you can then pass as a whole ($arguments = 'bar', 'baz'; foo.exe $arguments) - beware the issues around passing empty-string arguments and arguments with embedded " chars., however - see #1995.

Start-Process, which allows you to pass the list of arguments for an external executable as a single string, is the wrong tool for synchronously invoking external programs that are console applications - as you've observed, you lose integration with PowerShell's streams, even if you run the process synchronously (-Wait) in the same console window (-NoNewWindow, implied on Unix-like platforms).

Michael Klement (@mklement0)
This is a very clear and succinct explanation. Thank you.

Do you know if there are ways of invoking scripts hosted online without using iex?
ie
DownloadString or BITs $scriptURL | Invoke-Expression

I guess you could download the script to a temp ps1 file, call that, then delete it. But I'm curious if there's a way to do it entirely in memory.

I guess you could download the script to a temp ps1 file, call that, then delete it. But I'm curious if there's a way to do it entirely in memory.

Executing a script saved to a temporary file does it entirely in memory.

Michael Klement (@mklement0)

Michael Klement (@mklement0) wrote:

The & operator does not evaluate PowerShell source code stored in a string the way that Invoke-Expression does.

It only accepts a command name or path as its operand (or, alternatively a script block), so for running external programs the arguments must still be passed separately.

Or any of the command *Info objects, or a module object and then a scriptblock, or ... well, you get the drift, there's actually a lot of things, but in all cases, the thing & applies to an an object treated literally, not parsed...

Christopher Yeleighton (@yecril71pl) writes:

Executing a script saved to a temporary file does it entirely in memory.

Huh? Are we being super pedantic here for a reason?
Yes, it executes in memory, because that's what execution means.
But the point he's making is that downloading it to file creates a file on disk which needs cleaning up ...

Michael Klement (@mklement0) also wrote, much earlier:

the need for this idiom would go away:

* if we had support for `Invoke-WebRequest` output in `Invoke-Command` (e.g.,
  `iwr https://aka.ms/install-powershell.ps1 | icm`), as proposed in [#5909 (comment)](https://github.com/PowerShell/PowerShell/issues/5909#issuecomment-461192202), and things would get even simpler if #8835 gets implemented.
* or, as an out-of-process alternative, if `pwsh -` worked properly with code piped to it (e.g.,
  `irm https://aka.ms/install-powershell.ps1 | pwsh`)  - see #3223.

Yes, obviously if we did either of those things, we would certainly remove the need for Invoke-Expression -- but what would be the point? If we did that, we would have simply transferred all the "inherent security risks" the OP reffered to into new places which would be much harder to flag in a code review or static analyzer.

P.S. Sean Wheeler (@sdwheeler) none of the examples I gave you made it into that "wiki" page. Note that I use "wiki" in quotes here, because those things are less editable than the source code in this repo...

Steve Lee (@SteveL-MSFT)

Yes, obviously if we did either of those things, we would certainly remove the need for Invoke-Expression

The primary reason for not using Invoke-Expression (iex) is that is is inadequate for running the content of script files directly from the web:

The code runs directly in the caller's scope - a pollution of that scope that is likely unwanted.
If the downloaded code contains an exit statement, the session exits as a whole.
Passing arguments isn't supported.
(There are also 2 bugs, though they obviously could and should be fixed: Invoke-Expression validates nonexistent parameter default values. #8778 and Invoke-Expression doesn't make $PSCmdlet available in advanced-function scriptblocks #8815)

Rather than extend Invoke-Expression, it makes more sense to enhance Invoke-Command, as proposed.

Invoke-Command (icm) runs code in a child scope by default, while allowing running directly in the caller's scope as an opt-in via -NoNewScope.
With an Invoke-WebRequest (iwr) output object as input, it would have to treat exit as simply exiting the child scope, i.e. it would have to treat the code as if it were a script file.
It already supports passing arguments via -ArgumentList / -Args.
- However, since this is an array parameter, the syntax for multiple arguments is somewhat awkward, so making -ArgumentList a ValueFromRemainingParameters parameter should help.

# WISHFUL THINKING
iwr https://aka.ms/install-powershell.ps1 | icm -- -Preview"

Also, to recap Steve Lee (@SteveL-MSFT)'s workaround for use of Invoke-Expression (in combination with Invoke-RestMethod / irm), which works around the argument-passing issue and the scope issue, but not the exit issue:

iex "& { $(irm https://aka.ms/install-powershell.ps1) } -Preview"

Needless to say, even if that fully worked as intended, it would neither be obvious nor convenient.

Executing a script saved to a temporary file does it entirely in memory.

Huh? Are we being super pedantic here for a reason?
Yes, it executes in memory, because that's what execution means.
But the point he's making is that downloading it to file creates a file on disk which needs cleaning up ...

The storage for (small) temporary files is in RAM.

those things are less editable than the source code in this repo

The wiki is easier to edit when you are a member of the project.

@PowerShell/powershell-committee reviewed this 5/11/2022
We definitely do not want to deprecate Invoke-Expression - when you need it, you really need it. We also have no plans to remove it from the product, so marking it as deprecated which implies that it won't always be there is misleading. However, we do agree that we could possibly provide more guidance and information as to why it's such a dangerous tool.

Sean Wheeler (@sdwheeler)

Sean Wheeler (@sdwheeler) Here's my use case: I have a collection of ps files used for scripting sql db metadata, etc. that I usually run individually. I have one script that loops through all these files and calls them in order dynamically using Invoke-Expression. It would be cumbersome to convert the code within these files to functions, and I'd like to call them dynamically anyway. Is there another way to do this without iex?

rpressman

rpressman It sounds like you just want to call the scripts. You can use & or Invoke-Command in your loop instead.

I opened this issue to have a discussion about the idea. This really should have been a Discussion thread, but Discussions were not enabled at the time.

The question has been adequately answered. I don't think we should deprecate Invoke-Expression. However, we should probably have better guideance for the risks involved with using it. The article Preventing script injection attacks is a start.

Navigation Menu

Repository navigation

Should we officially "deprecate" Invoke-Expression? #11662

Summary of the new feature/enhancement

36 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Should we officially "deprecate" Invoke-Expression? #11662

Description

Summary of the new feature/enhancement

Activity

mklement0 commented on Jan 23, 2020

lukeb1961 commented on Jan 23, 2020

doctordns commented on Jan 23, 2020

danijeljw commented on Jan 23, 2020

sdwheeler commented on Jan 23, 2020

rjmholt commented on Jan 23, 2020

sdwheeler commented on Jan 23, 2020

SeeminglyScience commented on Jan 23, 2020

mklement0 commented on Jan 23, 2020

rjmholt commented on Jan 23, 2020

rjmholt commented on Jan 23, 2020

rjmholt commented on Jan 23, 2020

36 remaining items

aescolastico commented on Dec 16, 2020

yecril71pl commented on Dec 16, 2020

Jaykul commented on Dec 17, 2020

mklement0 commented on Dec 17, 2020

mklement0 commented on Dec 18, 2020

yecril71pl commented on Dec 20, 2020

JamesWTruher commented on May 12, 2022

rpressman commented on Jun 15, 2023

sdwheeler commented on Jun 15, 2023

sdwheeler commented on Jun 15, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions