Skip to content

Assertion failed 'nonVarPtrRegs == RBM_NONE' #37223

Closed
#40123
#37679
Closed
Assertion failed 'nonVarPtrRegs == RBM_NONE'#37223
#40123
#37679
Assignees
AndyAyersMS
Labels
JitStressCLR JIT issues involving JIT internal stress modesarch-arm32area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windows
Milestone
5.0.0

Description

@BruceForstall
BruceForstall
opened on May 31, 2020

Failure in libraries System.Threading.Tasks.Dataflow.Tests: release libraries, checked coreclr with:

COMPlus_TieredCompilation=0
COMPlus_JitStress=2
COMPlus_JitStressRegs=4

Windows arm32

E.g.,

build -s libs+libs.tests -runtimeConfiguration checked -c Release -arch arm /p:ArchiveTests=true

(the /p:ArchiveTests=true is required to build the RunTests.cmd wrapper scripts)

c:\bugs\System.Threading.Tasks.Dataflow.Tests\net5.0-Release>call .\RunTests.cmd -r C:\bugs\testhost\net5.0-Windows_NT-Release-arm
----- start Sun 05/31/2020 13:04:02.26 ===============  To repro directly: =====================================================
pushd c:\bugs\System.Threading.Tasks.Dataflow.Tests\net5.0-Release\
"C:\bugs\testhost\net5.0-Windows_NT-Release-arm\dotnet.exe" exec --runtimeconfig System.Threading.Tasks.Dataflow.Tests.runtimeconfig.json --depsfile System.Threading.Tasks.Dataflow.Tests.deps.json xunit.console.dll System.Threading.Tasks.Dataflow.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================

c:\bugs\System.Threading.Tasks.Dataflow.Tests\net5.0-Release>"C:\bugs\testhost\net5.0-Windows_NT-Release-arm\dotnet.exe" exec --runtimeconfig System.Threading.Tasks.Dataflow.Tests.runtimeconfig.json --depsfile System.Threading.Tasks.Dataflow.Tests.deps.json xunit.console.dll System.Threading.Tasks.Dataflow.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
  Discovering: System.Threading.Tasks.Dataflow.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Threading.Tasks.Dataflow.Tests (found 290 of 297 test cases)
  Starting:    System.Threading.Tasks.Dataflow.Tests (parallel test collections = on, max threads = 8)
Regset after BB27 gcr=0001 {r0}, byr=0000 {}, regVars=0004 {r2}

Assert failure(PID 12028 [0x00002efc], Thread: 6680 [0x1a18]): Assertion failed 'nonVarPtrRegs == RBM_NONE' in 'System.Collections.Generic.Dictionary`2[Int64,KeyValuePair`2][System.Int64,System.Collections.Generic.KeyValuePair`2[System.Boolean,System.__Canon]]:Remove(long):bool:this' during 'Generate code' (IL size 356)

    File: C:\gh\runtime\src\coreclr\src\jit\codegenlinear.cpp Line: 509
    Image: C:\bugs\testhost\net5.0-Windows_NT-Release-arm\dotnet.exe

----- end Sun 05/31/2020 13:04:13.92 ----- exit code -1073740286 ----------------------------------------------------------

category:correctness
theme:elt-hooks
skill-level:intermediate
cost:medium

Activity

  1. BruceForstall
    added
    arch-arm32
    os-windows
    JitStressCLR JIT issues involving JIT internal stress modes
    on May 31, 2020
  2. BruceForstall
    added this to the 5.0 milestone on May 31, 2020
  3. Dotnet-GitSync-Bot
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    untriagedNew issue has not been triaged by the area owner
    on May 31, 2020
  4. BruceForstall
    removed
    untriagedNew issue has not been triaged by the area owner
    on May 31, 2020

  5. BruceForstall commented on May 31, 2020

    @BruceForstall
    BruceForstall
    on May 31, 2020
    ContributorAuthor

    @CarolEidt

  6. BruceForstall
    assigned
    CarolEidt
    on Jun 1, 2020
  7. CarolEidt
    closed this as completedin #37679on Jun 12, 2020
  8. BruceForstall
    reopened this on Jul 18, 2020
  9. BruceForstall
    assigned
    AndyAyersMS
    and unassigned
    CarolEidt
    on Jul 18, 2020

  10. BruceForstall commented on Jul 18, 2020

    @BruceForstall
    BruceForstall
    on Jul 18, 2020
    ContributorAuthor

    After #39335, this is back:

    https://dev.azure.com/dnceng/public/_build/results?buildId=736732&view=ms.vss-test-web.build-test-results-tab&runId=22818242&resultId=173953&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

  11. CarolEidt commented on Jul 20, 2020

    @CarolEidt
    CarolEidt
    on Jul 20, 2020
    Contributor

    @AndyAyersMS - the failure can be duplicated using the arm altjit on 32-bit Windows. Not all the methods behave nicely with the altjit, but if you set COMPlus_AltJit=Remove (and COMPlus_AltJitName=protononjit.dll plus the variables mentioned above), you can duplicate the assert.

  12. 2 remaining items

  13. AndyAyersMS commented on Jul 21, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 21, 2020
    Member

    Issue seems to be that R2 is considered live throughout a return block as it holds a kept-alive this. Thus when armarch's
    genProfilingLeaveCallback tries to update R2's GC liveness to be non-gc , the update is ignored by gcMarkRegSetNpt as it will not modify live registers to be non-gc. So R2 remains gc live and hence the assert.

    Seems like the fix is that for arm, we need to make sure kept-alive this intervals interfere with the profiler scratch reg in return blocks, if we're adding a profiler exit hook, so that R2 is not live at the point we want to insert the leave callback, and the this pointer is still properly reported live at that point.

  14. AndyAyersMS commented on Jul 21, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 21, 2020
    Member

    Relevant part of the jitdump, during codegen, with a bit of extra logging I added. Note after "Moving R0 to R2 around leave profiler callback" there is no GC update for R2, it remains GC live, and that V00 == R2 is live throughout.

    =============== Generating BB27 [???..???) (return), preds={BB16,BB22} succs={} flags=0x00000000.40030050: keep internal label target LIR 
BB27 IN (2)={V00 V31}
     OUT(1)={V00    }

Recording Var Locations at start of BB27
  V00(STK->r2)  V31(r10)
Change life 0000000000000004 {V00} -> 0000000000002004 {V00 V31}
							V31 in reg r10 is becoming live  [------]
							Live regs: 0000 {} => 0400 {r10}
							Live regs: 0400 {r10} => 0404 {r2 r10}
							Removed GCVars: {V00}
							GC regs: 0000 {} => 0004 {r2}
							Byref regs: (unchanged) 0000 {}

      L_M63207_BB27:

      G_M63207_IG280:        ; offs=0003ACH, funclet=00, bbWeight=0   
Label: IG281, GCvars=0000000000000000 {}, gcrefRegs=0004 {r2}, byrefRegs=0000 {}

Scope info: begin block BB27, IL range [???..???)
Scope info: ignoring block beginning
Added IP mapping: NO_MAP STACK_EMPTY (G_M63207_IG281,ins#0,ofs#0) label
Generating: N761 (  1,  1) [001035] -------N-----     t1035 =    LCL_VAR   int    V31 tmp22        u:1 r10 (last use) REG r10 $413
                                                               /--*  t1035  int    
Generating: N763 (  2,  2) [001034] -------------     t1034 = *  CAST      int <- bool <- int REG r0 $7cb
							V31 in reg r10 is becoming dead  [001035]
							Live regs: 0404 {r2 r10} => 0004 {r2}
							Live vars: {V00 V31} => {V00}
IN0118:         4B      uxtb    r0, r10
                                                               /--*  t1034  int    
Generating: N765 (  3,  3) [001033] -------------             *  RETURN    int    REG NA $2e1
Moving R0 to R2 around leave profiler callback
      G_M63207_IG281:        ; offs=0003AEH, funclet=00, bbWeight=1   
IN0119:         2B      mov     r2, r0

      G_M63207_IG282:        ; offs=0003B2H, funclet=00, bbWeight=1   
IN011a:         4B      movw    r0, 0xff90

      G_M63207_IG283:        ; offs=0003B4H, funclet=00, bbWeight=1   
IN011b:         4B      movt    r0, 0x54d2

      G_M63207_IG284:        ; offs=0003B8H, funclet=00, bbWeight=1   
IN011c:         4B      movw    r12, 0xff90

      G_M63207_IG285:        ; offs=0003BCH, funclet=00, bbWeight=1   
IN011d:         4B      movt    r12, 0x54d2
NoGC Call: savedSet=57FF {r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r10 r12 lr}
Call: GCvars=0000000000000000 {}, gcrefRegs=0004 {r2}, byrefRegs=0000 {}

      G_M63207_IG286:        ; offs=0003C0H, funclet=00, bbWeight=1   
[286] Rec call GC vars = 0000000000000000
IN011e:         2B      blx     r12		// CORINFO_HELP_PROF_FCN_LEAVE
Moving R2 back to R0 after leave profiler callback
      G_M63207_IG287:        ; offs=0003C4H, funclet=00, bbWeight=1   
IN011f:         2B      mov     r0, r2
							GC regs: 0004 {r2} => 0005 {r0 r2}
Regset after BB27 gcr=0001 {r0}, byr=0000 {}, regVars=0004 {r2}

  15. AndyAyersMS commented on Jul 21, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 21, 2020
    Member

    it occurs to me that ensuring R2 was free was what #37679 was after, so I'll take a look at where that went wrong; perhaps it's a similar GC liveness transfer issue...?

  16. AndyAyersMS commented on Jul 29, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 29, 2020
    Member

    Simple repro:

    using System;
using System.Collections.Generic;

class X
{
    public static int Main()
    {
       var x = new System.Collections.Generic.Dictionary<long, KeyValuePair<bool, string>>();

       x[0] = new KeyValuePair<bool, string>(true, "hi");

       x.Remove(0);

       return 100;
    }
}

    On linux arm32, run with

    export COMPlus_TieredCompilation=0
export COMPlus_JitStressRegs=4
export COMPlus_JitStress=2

  17. CarolEidt commented on Jul 29, 2020

    @CarolEidt
    CarolEidt
    on Jul 29, 2020
    Contributor

    it occurs to me that ensuring R2 was free was what #37679 was after, so I'll take a look at where that went wrong

    Yes, that's what I was trying to accomplish, and I was surprised that it went wrong, since I didn't think that codegen paid attention to that killset. Another option to accomplish the same thing might be to allocate an internal register, and force it to use R2. That will cause it to interfere with anything else live at that point. However, I'm still not sure how what was wrong with the kill approach.

  18. AndyAyersMS commented on Jul 29, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 29, 2020
    Member

    Working on this now ... have restored your change and about to debug into one of the failures it causes.

  19. AndyAyersMS commented on Jul 30, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 30, 2020
    Member

    With #37679 restored, I can get JIT/Directed/refbyref/ref2iu_il_d/ref2iu_il_d.dll to crash readily under jitstress/gcstress as in the top message here. One method that has issues is MethodInfo.CreateDelegate.

    Here's the relevant bit of the codegen dump. It appears r2 gets GC killed too early; it needs to remain gc-live across the call.

    G_M35063_IG20:        ; func=00, offs=000062H, size=0002H, gcrefRegs=0001 {r0}, byrefRegs=0000 {}, byref
gcrReg +[r2]
IN0013: 000062  4602           mov     r2, r0
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG21:        ; func=00, offs=000064H, size=0004H, extend
gcrReg -[r0]
IN0014: 000064  F24B 3035      movw    r0, 0xb335
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG22:        ; func=00, offs=000068H, size=0004H, extend
IN0015: 000068  F2CF 0072      movt    r0, 0xf072
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG23:        ; func=00, offs=00006CH, size=0004H, extend
IN0016: 00006C  F24B 3C35      movw    r12, 0xb335
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG24:        ; func=00, offs=000070H, size=0004H, extend
IN0017: 000070  F2CF 0C72      movt    r12, 0xf072
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG25:        ; func=00, offs=000074H, size=0002H, extend
New gcrReg live regs=0000 {}
gcrReg -[r2]
IN0018: 000074  47E0           blx     r12		// CORINFO_HELP_PROF_FCN_LEAVE
						;; bbWeight=1    PerfScore 1.00
G_M35063_IG26:        ; func=00, offs=000076H, size=0002H, extend
gcrReg +[r0]
IN0019: 000076  4610           mov     r0, r2                  // ** gc stress crashes here **

  20. AndyAyersMS commented on Jul 30, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 30, 2020
    Member

    Suspect it's because we've added R2 to RBM_PROFILER_LEAVE_TRASH to avoid having LSRA allocate it across the call, but this also has the side effect of killing the GC ref in R2 when we do gc liveness.

    So we can either forcefully revive R2 in our custom handling (armarch's genProfilingLeaveCallback) or else special case this in lsra. Will try the former to see if it works.

  21. AndyAyersMS commented on Jul 30, 2020

    @AndyAyersMS
    AndyAyersMS
    on Jul 30, 2020
    Member

    Seems like #37679 plus

    --- a/src/coreclr/src/jit/emit.cpp
+++ b/src/coreclr/src/jit/emit.cpp
@@ -7692,7 +7692,12 @@ regMaskTP emitter::emitGetGCRegsKilledByNoGCCall(CorInfoHelpFunc helper)
             break;
 
         case CORINFO_HELP_PROF_FCN_LEAVE:
+#if defined(TARGET_ARMARCH)
+            // profiler scratch remains gc live
+            result = RBM_PROFILER_LEAVE_TRASH & ~RBM_PROFILER_RET_SCRATCH;
+#else
             result = RBM_PROFILER_LEAVE_TRASH;
+#endif
             break;
 
         case CORINFO_HELP_PROF_FCN_TAILCALL:

    will do the trick.

  22. AndyAyersMS
    closed this as completedin #40123on Jul 30, 2020
  23. AndyAyersMS
    added a commit that references this issue on Jul 30, 2020

    For arm32, kill REG_PROFILER_RET_SCRATCH for LSRA but not for GC (#40123

    32ddcd3
  24. Jacksondr5
    added a commit that references this issue on Aug 10, 2020

    For arm32, kill REG_PROFILER_RET_SCRATCH for LSRA but not for GC (dot…

    39e7542
  25. ghost locked as resolved and limited conversation to collaborators on Dec 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

JitStressCLR JIT issues involving JIT internal stress modesarch-arm32area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windows

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions