Skip to content

Stack overflow in crossgen2 #70385

Closed
#71135
Closed
Stack overflow in crossgen2#70385
#71135
Labels
arch-arm64area-ReadyToRunos-macosmacOS aka OSX
Milestone
7.0.0

Description

@filipnavara
filipnavara
opened on Jun 7, 2022

Tried this 3 times on a clean checkout (commit 08b3170) on macOS 12.3.1 (M1 ARM host):

  • Install prerequisites through brew
  • Run ./build.sh
  • Get this:
...
  ILCompiler.ReadyToRun -> /Users/filipnavara/Projects/runtime/artifacts/bin/ILCompiler.ReadyToRun/arm64/Debug/ILCompiler.ReadyToRun.dll
  crossgen2 -> /Users/filipnavara/Projects/runtime/artifacts/bin/coreclr/OSX.arm64.Debug/crossgen2/osx-arm64/crossgen2.dll
  crossgen2 -> /Users/filipnavara/Projects/runtime/artifacts/bin/coreclr/OSX.arm64.Debug/crossgen2/osx-arm64/publish/
  Stack overflow.
  Repeat 669 times:
  --------------------------------
     at System.NullReferenceException..ctor()
  --------------------------------
     at System.Runtime.Serialization.SerializationInfoEnumerator.get_Value()
     at System.Runtime.Loader.AssemblyLoadContext.OnAssemblyResolve(System.Reflection.RuntimeAssembly, System.String)
     at System.AppContext.Setup(Char**, Char**, Int32)
  /var/folders/q2/c94xfmgj35nd9289d04dhy480000gn/T/tmp15316b5ed3bc46b0919dfb735339accd.exec.cmd: line 2: 65532 Abort trap: 6           /Users/filipnavara/Projects/runtime/artifacts/bin/coreclr/OSX.arm64.Debug/crossgen2/osx-arm64/publish/crossgen2 /Users/filipnavara/Projects/runtime/artifacts/bin/coreclr/OSX.arm64.Debug/IL/System.Private.CoreLib.dll --out /Users/filipnavara/Projects/runtime/artifacts/obj/Microsoft.NETCore.App.Crossgen2/Debug/net7.0/osx-arm64/S.P.C.tmp

Activity

  1. filipnavara
    added
    arch-arm64
    os-macosmacOS aka OSX
    on Jun 7, 2022
  2. ghost added
    area-ReadyToRun
    on Jun 7, 2022
  3. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Jun 7, 2022

  4. EgorBo commented on Jun 7, 2022

    @EgorBo
    EgorBo
    on Jun 7, 2022
    Member

    Reproduces on mine (m1 mac mini, macOS 12.3) as well:

    Stack overflow.

  Assert failure(PID 26945 [0x00006941], Thread: 99227 [0x1839b]): Consistency check failed: FAILED: m_crawl.pFrame->IsTransitionToNativeFrame()
      File: /Users/egorbo/prj/runtime/src/coreclr/vm/stackwalk.cpp Line: 2274
      Image: /Users/egorbo/prj/runtime/artifacts/bin/coreclr/OSX.arm64.Debug/crossgen2/osx-arm64/publish/crossgen2

    @dotnet/crossgen-contrib

  5. filipnavara commented on Jun 7, 2022

    @filipnavara
    filipnavara
    on Jun 7, 2022
    MemberAuthor

    Yep, I got both of these messages on different runs (Consistency check failed first, followed by clean, and then I got the one in the original post).

  6. EgorBo commented on Jun 7, 2022

    @EgorBo
    EgorBo
    on Jun 7, 2022
    Member

    From what I see it happens only on "published" crossgen (even when it's invoked without arguments) - doesn't reproduce if I run it from sources via dotnet run

  7. EgorBo commented on Jun 7, 2022

    @EgorBo
    EgorBo
    on Jun 7, 2022
    Member

    I assume crossgen does not use NativeAOT on macos-arm64?

  8. trylek commented on Jun 7, 2022

    @trylek
    trylek
    on Jun 7, 2022
    Member

    I believe that's what @agocke told me the last time, on OSX we don't use NativeAOT.

  9. mangod9
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jun 8, 2022
  10. mangod9
    added this to the 7.0.0 milestone on Jun 8, 2022

  11. mangod9 commented on Jun 8, 2022

    @mangod9
    mangod9
    on Jun 8, 2022
    Member

    assume this repros only on macOS 12.3 and not earlier versions?

  12. filipnavara commented on Jun 8, 2022

    @filipnavara
    filipnavara
    on Jun 8, 2022
    MemberAuthor

    assume this repros only on macOS 12.3 and not earlier versions?

    I assume it reproduces on older versions too. In fact, 12.3 is not even the newest, I was just lazy to update.

  13. agocke commented on Jun 8, 2022

    @agocke
    agocke
    on Jun 8, 2022
    Member

    I believe that's what @agocke told me the last time, on OSX we don't use NativeAOT.

    Correct, this is R2R single-file.

  14. jakobbotsch commented on Jun 15, 2022

    @jakobbotsch
    jakobbotsch
    on Jun 15, 2022
    Member

    I am seeing a somewhat similar problem on windows when using a recent SDK from dotnet/installer (7.0.100-preview.6.22313.7) and replacing the bits with checked clrjit.dll, coreclr.dll and SPC.dll from a recent build. On dotnet.exe new console I then hit the same stack overflow and assertion that @EgorBo wrote above.

    In debugger the stack trace looks like:

    0:000> kn50
 # Child-SP          RetAddr               Call Site
00 0000004c`1b676ff0 00007ffe`7259f4a8     coreclr!TypeKey::GetModule+0xa6 [C:\dev\dotnet\runtime\src\coreclr\vm\typekey.h @ 144] 
01 0000004c`1b6770b0 00007ffe`725b311e     coreclr!ClassLoader::ComputeLoaderModule+0xd8 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 205] 
02 0000004c`1b6771a0 00007ffe`725b3aff     coreclr!ClassLoader::LookupInLoaderModule+0x32e [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 1079] 
03 0000004c`1b677330 00007ffe`725a918c     coreclr!ClassLoader::LookupTypeHandleForTypeKey+0x31f [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 1116] 
04 0000004c`1b6774a0 00007ffe`725aa4d8     coreclr!ClassLoader::LoadConstructedTypeThrowing+0xcec [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 954] 
05 0000004c`1b677850 00007ffe`7270182a     coreclr!ClassLoader::LoadGenericInstantiationThrowing+0x7e8 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 1683] 
06 0000004c`1b677b20 00007ffe`72712a66     coreclr!MethodTable::InterfaceMapIterator::GetInterface+0x69a [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8652] 
07 0000004c`1b677dd0 00007ffe`726f76db     coreclr!MethodTable::VerifyThatAllVirtualStaticMethodsAreImplemented+0x176 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8271] 
-------------------------------------------------------------
08 0000004c`1b677f50 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x143b [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4697] 
09 0000004c`1b678130 00007ffe`726f6b85     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
0a 0000004c`1b678200 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x8e5 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4501] 
0b 0000004c`1b6783e0 00007ffe`725b5724     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
0c 0000004c`1b6784b0 00007ffe`725aefc1     coreclr!PushFinalLevels+0x114 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 3186] 
0d 0000004c`1b6785b0 00007ffe`725a942a     coreclr!ClassLoader::LoadTypeHandleForTypeKey+0x251 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 3247] 
0e 0000004c`1b678740 00007ffe`725aa4d8     coreclr!ClassLoader::LoadConstructedTypeThrowing+0xf8a [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 987] 
0f 0000004c`1b678af0 00007ffe`7270182a     coreclr!ClassLoader::LoadGenericInstantiationThrowing+0x7e8 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 1683] 
10 0000004c`1b678dc0 00007ffe`726fa6c2     coreclr!MethodTable::InterfaceMapIterator::GetInterface+0x69a [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8652] 
11 0000004c`1b679070 00007ffe`7270f141     coreclr!MethodTable::FindDefaultInterfaceImplementation+0xe82 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 5667] 
12 0000004c`1b6795e0 00007ffe`72712eda     coreclr!MethodTable::ResolveVirtualStaticMethod+0x821 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8080] 
13 0000004c`1b6797f0 00007ffe`726f76db     coreclr!MethodTable::VerifyThatAllVirtualStaticMethodsAreImplemented+0x5ea [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8280] 
-------------------------------------------------------------
14 0000004c`1b679970 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x143b [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4697] 
15 0000004c`1b679b50 00007ffe`726f6b85     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
16 0000004c`1b679c20 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x8e5 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4501] 
17 0000004c`1b679e00 00007ffe`725b5724     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
18 0000004c`1b679ed0 00007ffe`725aefc1     coreclr!PushFinalLevels+0x114 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 3186] 
19 0000004c`1b679fd0 00007ffe`725a942a     coreclr!ClassLoader::LoadTypeHandleForTypeKey+0x251 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 3247] 
1a 0000004c`1b67a160 00007ffe`725aa4d8     coreclr!ClassLoader::LoadConstructedTypeThrowing+0xf8a [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 987] 
1b 0000004c`1b67a510 00007ffe`7270182a     coreclr!ClassLoader::LoadGenericInstantiationThrowing+0x7e8 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 1683] 
1c 0000004c`1b67a7e0 00007ffe`726fa6c2     coreclr!MethodTable::InterfaceMapIterator::GetInterface+0x69a [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8652] 
1d 0000004c`1b67aa90 00007ffe`7270f141     coreclr!MethodTable::FindDefaultInterfaceImplementation+0xe82 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 5667] 
1e 0000004c`1b67b000 00007ffe`72712eda     coreclr!MethodTable::ResolveVirtualStaticMethod+0x821 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8080] 
1f 0000004c`1b67b210 00007ffe`726f76db     coreclr!MethodTable::VerifyThatAllVirtualStaticMethodsAreImplemented+0x5ea [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 8280] 
-------------------------------------------------------------
20 0000004c`1b67b390 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x143b [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4697] 
21 0000004c`1b67b570 00007ffe`726f6b85     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
22 0000004c`1b67b640 00007ffe`727bd3e3     coreclr!MethodTable::DoFullyLoad+0x8e5 [C:\dev\dotnet\runtime\src\coreclr\vm\methodtable.cpp @ 4501] 
23 0000004c`1b67b820 00007ffe`725b5724     coreclr!TypeHandle::DoFullyLoad+0x173 [C:\dev\dotnet\runtime\src\coreclr\vm\typehandle.cpp @ 1212] 
24 0000004c`1b67b8f0 00007ffe`725aefc1     coreclr!PushFinalLevels+0x114 [C:\dev\dotnet\runtime\src\coreclr\vm\clsload.cpp @ 3186]

    It does not reproduce with #66887 reverted, though I hit other issues then so maybe the SDK just does not get to the problematic part. But cc @trylek anyway.

    EDIT: After a fresh merge from main I no longer hit this problem, so maybe it is already resolved. The commit I was hitting the problem on was 2c110b6.

  15. trylek commented on Jun 15, 2022

    @trylek
    trylek
    on Jun 15, 2022
    Member

    It does seem related to my change, I'll take a look, thanks Jakob for the heads-up.

  16. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Jun 22, 2022
  17. trylek
    closed this as completedin #71135on Jun 22, 2022
  18. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Jun 22, 2022
  19. ghost locked as resolved and limited conversation to collaborators on Jul 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    arch-arm64area-ReadyToRunos-macosmacOS aka OSX

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions