Repository navigation
Optimize library code using arm64 intrinsics #33308
Description
Activity
System.TextSystem.Text.ASCIIUtility,System.Text.UnicodeandSystem.SpanHelpersare the most important ones for Web workloads.@jkotas It's my understanding that ASP.NET is currently not running benchmarks on arm64. Hopefully that will change, then we'd be able to see benefits from optimizing these.
Also, BitArray is chosen to be first to implement because it's simple, has benchmarks defined, and can easily be used as a proof-of-concept of the basics of the intrinsics implementation -- not because it's the most important class.
CC. @CarolEidt, @echesakovMSFT, @GrabYourPitchforks, @TamarChristinaArm
Would love to help this! Though, I remember majority of the ARM intrinsics being unavailable last time I looked up (only available via experimental NuGet package). Has this been addressed? I have seen a few API review sessions where ARM intrinsics are discussed but I don't know if they are implemented.
One concern is that I can't quite run tests and benchmarks locally since I do not own an ARM machine (well, I do - but it's an RPi 3B+). I know those exist as part of CI/CD but are the perf numbers available to non-MSFT people?
I have seen a few API review sessions where ARM intrinsics are discussed but I don't know if they are implemented.
The implementation is in progress, so there will be feature gaps.
One concern is that I can't quite run tests and benchmarks locally since I do not own an ARM machine (well, I do - but it's an RPi 3B+). I know those exist as part of CI/CD but are the perf numbers available to non-MSFT people?
That's a bigger problem. The CI will do feature testing on ARM64, but there is currently no perf testing available to non-MSFT (and minimal available internal). You probably don't want to depend on the CI to do all functional testing for you. I think it's possible to use the RPi for this if you install, say, Ubuntu 64-bit.
I think it's possible to use the RPi for this if you install, say, Ubuntu 64-bit.
I have successfully run perf tests on a RPi 4 with Ubuntu 64-bit; I assume it would also work on the RPi 3B+.
Reacted by Adeel MujahidReacted by Ganbarukamo41System.Numerics.Intrinsics #33496, this (and the namespaces under it) should beSystem.Runtime.Intrinsics.Reacted by Bruce Forstall17 remaining items
@tannergooding That's great to hear. The deadline is whatever your team chooses as the deadline, for how you manage 5.0 work. The CLR CodeGen team doesn't expect to have enough time/resources to these ourselves for 5.0 (we chose to do lots of this libraries work initially, to validate the intrinsics work and seed the optimizations), but certainly other teams/people can do them instead.
Reacted by Tanner Gooding and Jeff HandleyWhat is the current deadline for getting these changes in by?
The platform teams (us) are aiming to be feature complete by Preview 8 snap which is July 15 (internal link)
@BruceForstall - we'll keep you updated on our planning and progress after we kick off the work next Monday.
@tannergooding what is our test strategy for all these -- presumably we're just relying on our regular test bed and having enough hardware variation.
In particular, how do we get coverage on the software fallback paths? In the past, we've said that the ARM machines cover this path for us.
@danmosemsft We have two AzDO pipelines for stressing the ISAs, which includes disabling hardware intrinsics:
runtime-coreclr jitstress-isas-arm: https://dev.azure.com/dnceng/public/_build?definitionId=665
runtime-coreclr jitstress-isas-x86: https://dev.azure.com/dnceng/public/_build?definitionId=664These are defined using the following modes (in eng\pipelines\common\templates\runtimes
\run-test-job.yml, with the tags defined in src\coreclr\tests\testenvironment.proj):${{ if in(parameters.testGroup, 'jitstress-isas-arm') }}: scenarios: - jitstress_isas_incompletehwintrinsic - jitstress_isas_nohwintrinsic - jitstress_isas_nohwintrinsic_nosimd - jitstress_isas_nosimd ${{ if in(parameters.testGroup, 'jitstress-isas-x86') }}: scenarios: - jitstress_isas_incompletehwintrinsic - jitstress_isas_nohwintrinsic - jitstress_isas_nohwintrinsic_nosimd - jitstress_isas_nosimd - jitstress_isas_x86_noaes - jitstress_isas_x86_noavx - jitstress_isas_x86_noavx2 - jitstress_isas_x86_nobmi1 - jitstress_isas_x86_nobmi2 - jitstress_isas_x86_nofma - jitstress_isas_x86_nohwintrinsic - jitstress_isas_x86_nolzcnt - jitstress_isas_x86_nopclmulqdq - jitstress_isas_x86_nopopcnt - jitstress_isas_x86_nosse - jitstress_isas_x86_nosse2 - jitstress_isas_x86_nosse3 - jitstress_isas_x86_nosse3_4 - jitstress_isas_x86_nosse41 - jitstress_isas_x86_nosse42 - jitstress_isas_x86_nossse3@tannergooding can comment on whether that is still sufficient for x86/x64, arm, as well as scenarios like R2R. Note that ARM32 doesn't support hardware intrinsics, so requires the fallback path.
@jeffhandley is work planned for 5.0 complete? If so please let's close this now.
@danmosemsft - There are still some APIs under
System.Text.ASCIIUtilityandSystem.Buffersthat are not optimized. You can check the status of those methods in the PR description.@danmosemsft I'm aggregating the status of these and will make a decision today whether the remaining items should be moved to 6.0.
Reacted by Dan MoseleyHere's the list of intrinsics efforts not merged into
release/5.0. I recommend that we move them all to 6.0.- System.Buffers Optimize System.Buffers for arm64 using cross-platform intrinsics #35033
- System.Buffers.Text.Base64.DecodeFromUtf8()
- System.Buffers.Text.Base64.EncodeToUtf8()
- System.Text.ASCIIUtility Optimize System.Text.ASCIIUtility using arm64 intrinsics #35034
- System.Text.ASCIIUtility.GetIndexOfFirstNonAsciiChar() - Open PR Get index of first non ascii char #39507
- System.Text.ASCIIUtility.NarrowUtf16ToAscii() - Open PR Narrow utf16 to ascii #39509
- System.Text.Json.JsonReaderHelper (from @benaadams)
- IndexOfOrLessThan() - Open PR Intrinsicify (Sse, Axv2, Arm64, wasm) JsonReaderHelper.IndexOfOrLessThan #41097
I will take the following actions:
- Update the text of this issue description for the Buffers and ASCIIUtility items to note that those will be completed in 6.0
- Move Optimize System.Buffers for arm64 using cross-platform intrinsics #35033 (Buffers) to the 6.0.0 milestone
- Comment on Optimize System.Text.ASCIIUtility using arm64 intrinsics #35034 (ASCIIUtility) which methods were not completed in 5.0.0, and then close that issue
- Create a new issue for the remaining ASCIIUtility methods
- System.Buffers Optimize System.Buffers for arm64 using cross-platform intrinsics #35033
This all looks good to me. Thanks @jeffhandley for summarizing and closing out the 5.0 release.
And thanks everyone for the great work with all these arm64 improvements!
Reacted by Jeff Handley and Julie LeeAgreed. And I think we can please close this then.
And thanks everyone for the great work with all these arm64 improvements!
+100 ! and appreciation for @BruceForstall for establishing clarity originally by creating this list.
Reacted by Jeff Handley and Julie Lee- ghost locked as resolved and limited conversation to collaborators
on Dec 10, 2020
The following classes/functions in the libraries have Intel x86/x64 intrinsics usage. These are where
_ISA_.IsSupported()is called. This information was collected manually and might not be complete. Some of these function names represent many overloads. There are some vectorized helper methods not shown here -- where a function callsIsSupportedand then calls a specific helper function to do the actual work, such as for SSE2 or AVX2 specifically. There are other cases whereVector<T>is used, but arm64 already supports that (it should be verified that the arm64Vector<T>code is complete and performant).When each of these has added an arm64-specific intrinsics optimization, it should be "checked off".
The sections below are ordered in the presumed priority order that they should be implemented in. (There is no assumed priority order for the individual functions in each section.)
It is expected that
System.Collections.BitArray,System.Numerics, andSystem.SpanHelperswill be "arm64 intrinsi-fied" for .NET 5. If possible,System.BuffersandSystem.Textwill as well, but that is not considered required.System.Collections.BitArray #33309
System.Runtime.Intrinsics #33496
Vector64
Vector128
Vector256
System.Numerics
System.Numerics.BitOperations #33495
System.Numerics.Matrix4x4 #33565
System.SpanHelpers #33707
[ ] System.SpanHelpers.SequenceCompareTo(byte)(SIMD vector implementation is fast enough)[ ] System.SpanHelpers.SequenceEqual(byte)(SIMD vector implementation is fast enough)[ ] System.SpanHelpers.LocateFirstFoundByte()(Only used by SIMD version ofIndexOfandIndexOfAnywhich are already optimized by ARM64 intrinsics)System.Buffers #35033
(Not completed in 5.0.0; moved to 6.0.0)
System.Text
System.Text.ASCIIUtility #35034
(Not completed in 5.0.0; #41292 contains the items moved to 6.0.0)
System.Text.Unicode #35035
System.Text.Encodings.Web #35036