Repository navigation
Accessing global functions from inside ForEach-Object -Parallel #12313
Description
Hello,
I am processing large amount of Graph calls using powershell script block - 500K in my case. I want to process script blocks in batches. My ForEach-Object loop is defined inside class, say Library, in method, say ExportItems. Library.ExportItems call global function named Log([string]text). Method can successfully use it, however Parallel block cannot.
Steps to reproduce
using module .\logger.ps1
class SPOLibrary {
[void]ExportItems (
[string]$OutputFileName
) {
try {
# MAIN: process loaded queue
$this.Process( $OutputFileName)
}
finally {
}
}
[void]Process (
[string]$OutputFileName
){
[int]$_batchSize = 20
1..$_batchSize | ForEach-Object -Parallel {
# NOW here is the global function that is not visible to this block
Log "text"
}
}Expected behavior
global functions should be visible within script block
Actual behavior
function is not visible
Environment data
Name Value
---- -----
PSVersion 7.0.0
PSEdition Core
GitCommitId 7.0.0
OS Microsoft Windows 10.0.19603
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
Activity
PaulHigin commentedon Apr 14, 2020
The ForEach-Object -Parallel feature uses PowerShell runspaces to run each loop iteration concurrently. PowerShell runspaces are isolation abstractions and don't provide a way to share state between instances.
However, there are many examples (including this) of reasonable expectations for the loop iteration state to be the same as what the ForEach-Object cmdlet runs in. One solution is to allow an option that replicates the state, at some cost to performance. We have a current issue to track this (so I'll close this one). #12240
Also, here is an article that goes into greater detail about how ForEach-Object -Parallel works. But note that one issue, runspace re-use, has been fixed in 7.1 preview (#12122).
https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/
devmanlab commentedon Apr 15, 2020
thanks