Repository navigation
Investigate what would be required to move some SIMD functionality out of the JIT and into managed code #102275
Description
Historically a number of types (such as Vector2, Vector3, and Vector4) have had their functionality implemented as Intrinsic in the JIT. This was done for many reasons, but primarily for performance of these types that are known to be used in perf critical workloads since the JIT needed to generate specialized instructions that were otherwise not accessible.
Since then, we've had a broad range of support exposed around platform specific hardware intrinsic APIs and it is now possible to implement many of these other APIs purely in managed code. However, this has not been done due to concerns that it may regress perf. This concern is particularly relevant in complex algorithms where the inliner may decide that it is out of budget and no longer tries to optimize this code.
It would be beneficial to actually prototype the work of moving some of these implementations out of the JIT and purely into managed code. This should let us see what in the JIT may still be getting pessimized and work towards fixing it. The long term benefits of moving the complexity into managed are then substantial:
- The JIT becomes simpler/smaller
- Multiple runtimes no longer need to do the work to optimize these additional APIs, they only need to support the xplat APIs
- Users can more readily debug the code being executed and contribute improvements themselves
- Such fixes/improvements are also much lower risk
- Pain points that end users may encounter become more readily identifiable to the team, allowing fixes to be provided
- etc
Activity
dotnet-policy-service commentedon May 15, 2024
Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.
tannergooding commentedon May 15, 2024
In an ideal world, we would be able to remove the entirety of
simdashwintrinsicfrom the JIT and much of the handling around the various xplat APIs inhwintrinsiclist*(essentially any NamedIntrinsic which is invalid to represent as a node ID).tannergooding commentedon May 15, 2024
CC. @EgorBo
EgorBo commentedon May 15, 2024
I fully support this 🙂
tannergooding commentedon May 15, 2024
I've assigned myself for the time being, but likely won't get to this until after we snap for .NET 9 RC1 and main opens for .NET 10
If someone else has some time to work on prototyping this sooner, please reach out and we can see about getting it assigned out instead.
tannergooding commentedon May 17, 2024
The prototype overall looked good, however I'm going to block the work on #11062 being resolved as the risk is too high otherwise. -- Not being intrinsic means inlining has to happen which means various constants can't be seen as constant until post import when various other optimizations (like forward sub, constant prop, etc) have happened. This really hurts a number of common existing code patterns and will regress user workloads.
There's some other aspects that likely should be investigated more as well, such as the overhead of the JIT needing to do more work (more inlining, more optimizations) to make things generate the same code and therefore likely reducing the throughput and increasing overhead of the JIT. But, I think those aspects do not need to be blocked on and can instead be handled incrementally.
tannergooding commentedon May 17, 2024
As described in #11062 (comment), we have support for rewriting some nodes (GT_INTRINSIC) back into a call as part of rationalization (the final step before lowering). This works fine when all inputs and return types are primitive.
However, the support does not currently exist to do the more complex fixups and ABI handling that would be required to convert a
GT_HWINTRINSIC(which may haveTYP_SIMD,TYP_MASK, or evenTYP_STRUCTinputs/outputs) back into a call, nor do we currently have a node capable of tracking all the required information (like the method handle) to make that transform possible.tannergooding commentedon May 18, 2024
I took another look at allowing
GT_HWINTRINSICto carry theCORINFO_METHOD_HANDLEdown to rationalization and tried to update things to handle it all.The big things to note are that such signatures require us to re-resolve the
CORINFO_SIG_INFOso we can get theCORINFO_CLASS_HANDLEfor each argument so we can correctlyNewCallArg::Struct(...)forTYP_SIMDoperands andfgMorphArgsappears to do basically all the right fixups and such for input parametersHowever, what's not handled is the ABI handling for
TYP_SIMDreturns which currently looks to be mostly isolated toimpFixupCallStructReturn(which changes it fromTYP_STRUCTtoTYP_SIMDand such) but there's then more handling which seems to be scattered about import as well that does the fixup to insertWellKnownArg::RetBuffer, change the return type to actually beTYP_VOID, allocate a local to hold the return buffer, and replaces the user of the call with the the return buffer so that the value can actually be consumedRefactoring the last stuff to not be
importcentric and to allow it to be shared with rationalization was a bit out of my depth; I tried hacking some bits together and got it mostly working, but its not something that would ever pass code review 😄Also worth noting that for input args it wasn't entirely clear whether
fgMorphArgswas doing the right thing with regards to shadow copying values, it looked like it might not have been and that was instead handled separately in import, so that may be another place that needs logic to be centralized so that rationalization (or even morph) could reuse it for these intrinsic calls that involve non primitive typestannergooding commentedon Aug 8, 2024
Closing this.
The investigation was done and it was decided that while non-core types like
Vector2/3/4,Plane,Quaternion, andMatrix4x4could be implemented in managed. We want to keep ABI primitive types likeVector<T>andVector64/128/256/512<T>accelerated directly in the JIT to ensure everything gets the necessary perf and can be relied upon to build the non-core types..NET 9 has correspondingly moved the implementation of types like
Vector2/3/4, etc entirely into managed.