Skip to content

PowerShell hangs when exiting having used a PowerShell runspace pool/runspace #5205

Closed
Closed
PowerShell hangs when exiting having used a PowerShell runspace pool/runspace#5205
Assignees
PaulHigin
Labels
Issue-BugIssue has been identified as a bug in the productWG-Enginecore PowerShell engine, interpreter, and runtime
Milestone
6.0.0-RC

Description

@TylerLeonhardt
TylerLeonhardt
opened on Oct 23, 2017

PowerShell Core hangs when exiting and using a PowerShell runspace pool. On Windows PowerShell it exits successfully.

Steps to reproduce

here's the gist for my code: https://gist.github.com/tylerl0706/2d87a13c6147278bb4e8565585aa9722

Steps:

PS > Add-Type -Path "path/to/TestHttpListener/bin/Debug/netstandard2.0/TestHttpListener.dll"
PS > $app = [TestHttpListener.TestHttpListener]::new()
PS > $app.Start(8081)
PS > Invoke-WebRequest -Uri http://localhost:8081 # VERY IMPORTANT
PS > exit

Expected behavior

PowerShell successfully exists

Actual behavior

PowerShell hangs and doesn't give the prompt back - this is causing my AppVeyor and Travis builds to fail due to timeout :(

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta.8
PSEdition                      Core
GitCommitId                    v6.0.0-beta.8
OS                             Darwin 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Activity

  1. TylerLeonhardt commented on Oct 23, 2017

    @TylerLeonhardt
    TylerLeonhardt
    on Oct 23, 2017
    MemberAuthor

    sergei (@vors) and I were both able to repro this locally.

  2. Ilya (iSazonov)
    added
    WG-Enginecore PowerShell engine, interpreter, and runtime
    Issue-BugIssue has been identified as a bug in the product
    on Oct 24, 2017

  3. SteveL-MSFT commented on Oct 25, 2017

    @SteveL-MSFT
    Steve Lee (SteveL-MSFT)
    on Oct 25, 2017
    Member

    I think I have a simpler repro without using httplistener

    PS /Users/steve> $rs = [runspacefactory]::CreateRunspacePool(1,5)                                                                                               
PS /Users/steve> $rs.Open()                                                                                                                                     
PS /Users/steve> $ps = [powershell]::Create()                                                                                                                   
PS /Users/steve> $ps.RunspacePool = $rs                                                                                                                         
PS /Users/steve> $null = $ps.AddScript({1+1})                                                                                                                           
PS /Users/steve> $ps.Invoke()
PS /Users/steve> exit #hangs
  4. Steve Lee (SteveL-MSFT)
    added this to the milestone on Oct 25, 2017

  5. PaulHigin commented on Oct 25, 2017

    @PaulHigin
    Paul Higinbotham (PaulHigin)
    on Oct 25, 2017
    Contributor

    This is hanging on the PipelineThread._workItemReady event wait. On exit this event should get set and release the thread wait via the finalizer:
    https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/hostifaces/LocalPipeline.cs#L1340

    But it looks like the finalizer is not being called.
    Dongbo Wang (@daxian-dbw) This appears to be a bug in the .Net Core CLR since it works in Windows PowerShell.

  6. Steve Lee (SteveL-MSFT)
    modified the milestones: , 6.0.0-RC on Nov 2, 2017

  7. TylerLeonhardt commented on Nov 3, 2017

    @TylerLeonhardt
    TylerLeonhardt
    on Nov 3, 2017
    MemberAuthor

    Is this something we should open on .NET Core CLR?

  8. SteveL-MSFT commented on Nov 3, 2017

    @SteveL-MSFT
    Steve Lee (SteveL-MSFT)
    on Nov 3, 2017
    Member

    @tylerl0706 need more data to provide to dotnetcore team before we hand it off to them

  9. powercode commented on Nov 3, 2017

    @powercode
    Staffan Gustafsson (powercode)
    on Nov 3, 2017
    Collaborator

    I don't think we can depend on a finalizer being run. We should invoke dispose explicitly if we depend on it to run. There are quite a few cases where finalizers are not run - see https://blogs.msdn.microsoft.com/oldnewthing/20100809-00/?p=13203/ and
    https://stackoverflow.com/questions/3458177/are-net-finalizers-always-executed.

  10. PaulHigin commented on Nov 3, 2017

    @PaulHigin
    Paul Higinbotham (PaulHigin)
    on Nov 3, 2017
    Contributor

    Sure, finalizers sometimes don't run due to various reasons, but in this case the finalizer never runs. Also this cleanup has relied on finalizers for years without any problems. So this appears to be a CLR regression.

    We can (and do) do some cleanup during exit, such as closing remote runspaces and disconnect running remote jobs. But this cleanup is not guaranteed to run in all cases same as the finalizer thread.

  11. PaulHigin commented on Nov 3, 2017

    @PaulHigin
    Paul Higinbotham (PaulHigin)
    on Nov 3, 2017
    Contributor

    Unless you are suggesting that there is a bug in PowerShell that is causing the finalizer thread not to run (such as throwing an unhandled exception on the thread). That would be a regression on our part. But with my experimentation I don't see any unhandled exceptions. It looks like the finalizer is just not getting called.

  12. powercode commented on Nov 3, 2017

    @powercode
    Staffan Gustafsson (powercode)
    on Nov 3, 2017
    Collaborator

    I'm saying that it is brittle. Any type a user loads with a misbehaving finalizer may cause this to happen.
    If we have control of these objects, (and the PipelineThread seems like it should fall in this category) we should also be able to write infrastructure to dispose of them.

  13. PaulHigin commented on Nov 3, 2017

    @PaulHigin
    Paul Higinbotham (PaulHigin)
    on Nov 3, 2017
    Contributor

    I understand. And we have that infrastructure on exit. My point is that anything we write is just as susceptible to catastrophic failures.

  14. anmenaga commented on Nov 3, 2017

    @anmenaga
    Andrew (anmenaga)
    on Nov 3, 2017

    Paul Higinbotham (@PaulHigin) I don't know... Yes, finalizer never getting called might be a CLR regression, but it feels like we should be able to set _workItemReady (e.g. by calling LocalPipeline:Dispose()) in our code without relying on the mercy of the garbage collector.

  15. PaulHigin commented on Nov 3, 2017

    @PaulHigin
    Paul Higinbotham (PaulHigin)
    on Nov 3, 2017
    Contributor

    Andrew (@anmenaga), We really need to be able to depend on the CLR to work as advertised. We have other places in the code that relies on the finalizer to work. I'll investigate more and if this looks like a CLR regression I'll notify the team.

    But for the short term I'll create a workaround to fix this particular hang problem for RC using our existing cleanup on exit code.

  16. Paul Higinbotham (PaulHigin)
    self-assigned this
    on Nov 3, 2017

  17. daxian-dbw commented on Nov 14, 2017

    @daxian-dbw
    Dongbo Wang (daxian-dbw)
    on Nov 14, 2017
    Member

    Close via #5356

  18. Dongbo Wang (daxian-dbw)
    closed this as completedon Nov 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Issue-BugIssue has been identified as a bug in the productWG-Enginecore PowerShell engine, interpreter, and runtime

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions