Add support for arm64 i.e. M1 mac - #29

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/how-does-scijava-platform-bits-expand-issues-on-m1-mac/62238/3

Thanks @paddyroddy!

I think this is a necessary addition, but unfortunately not sufficient to fully support native classifiers for M1 macs. 😢

Right now we have:

<scijava.natives.classifier>natives-${scijava.platform.family.medium}_${scijava.platform.bits}</scijava.natives.classifier>

But the scijava.platform.family.* properties are set for Windows, macOS, and Linux OSes... not for x86 versus ARM architecture. For that, one needs to use ${os.arch}. We may need to complexify the logic here. A while back I wrote up a bunch of the different common native classifier patterns I have seen in the wild—I'm going to track that down again and will follow up with a new issue here in pom-scijava-base.

So at the moment what would it do when running on Linux x86 vs AMD for example?

OK, so here is what I wrote in 42759cc:

We now define four platform family names: short, medium, long, longest. And we define a property scijava.natives.bits for bitness. If what you need is os.arch, you can reference ${os.arch} directly.

The scijava.natives.classifier property is now defined as ${scijava.natives.family.medium}_${scijava.natives.bits},
resulting in the following values for our five favorite platforms:

windows_32, windows_64, osx_64, linux_32, linux_64

If you need something else for your particular platform-specific classifier, you can define your own property. Examples:

<lwjgl.natives.classifier>natives-${scijava.natives.family.long}</lwjgl.natives.classifier>
<ffmpeg.natives.classifier>${scijava.natives.family.longest}-${os.arch}</ffmpeg.natives.classifier>

And then in the dependency, reference the corresponding property.

And the pom.xml currently has this comment about scijava.natives.classifier:

Classifier name for referencing platform-specific artifacts.
By default, this matches the scijava/native-lib-loader naming scheme.

And that naming scheme is currently:

windows_32, windows_64, osx_64, linux_32, linux_64, windows_arm64, linux_arm64

I.e. arm64, versus just 64 for implied x86 architecture.

I see two ways of addressing this here in pom-scijava-base:

1. Set scijava.platform.bits to arm64 instead of 64. Feels hacky, and potentially breaking some use cases of the bits value.
2. Introduce new properties scijava.platform.arch.short = arm or empty string (or potentially something else), and scijava.platform.arch.long = arm or x86 for completeness, and then change the scijava.natives.classifier to natives-${scijava.platform.family.medium}_${scijava.platform.arch.short}${scijava.platform.bits}.

Either option would preserve backwards compatibility, while supporting ARM better, such that we can begin building and deploying e.g. FLIMLib natives that work as desired with the native-lib-loader. What do you think, @paddyroddy?

So at the moment what would it do when running on Linux x86 vs AMD for example?

Right now, I think in either case you'd get a native classifier value of natives-linux_64, i.e. no difference, which is obviously not enough to support ARM.

Here are three examples of native library naming in practice:

JOGL

• jogl-all-2.3.2-natives-android-aarch64.jar
• jogl-all-2.3.2-natives-android-armv6.jar
• jogl-all-2.3.2-natives-linux-amd64.jar
• jogl-all-2.3.2-natives-linux-armv6.jar
• jogl-all-2.3.2-natives-linux-armv6hf.jar
• jogl-all-2.3.2-natives-linux-i586.jar
• jogl-all-2.3.2-natives-macosx-universal.jar
• jogl-all-2.3.2-natives-solaris-amd64.jar
• jogl-all-2.3.2-natives-solaris-i586.jar
• jogl-all-2.3.2-natives-windows-amd64.jar
• jogl-all-2.3.2-natives-windows-i586.jar

FFMPEG

• ffmpeg-3.4.1-1.4-android-arm.jar
• ffmpeg-3.4.1-1.4-android-x86.jar
• ffmpeg-3.4.1-1.4-linux-armhf.jar
• ffmpeg-3.4.1-1.4-linux-ppc64le.jar
• ffmpeg-3.4.1-1.4-linux-x86.jar
• ffmpeg-3.4.1-1.4-linux-x86_64.jar
• ffmpeg-3.4.1-1.4-windows-x86.jar
• ffmpeg-3.4.1-1.4-windows-x86_64.jar

OpenCV

• opencv-4.0.1-1.5-android-arm.jar
• opencv-4.0.1-1.5-android-arm64.jar
• opencv-4.0.1-1.5-android-x86.jar
• opencv-4.0.1-1.5-android-x86_64.jar
• opencv-4.0.1-1.5-ios-arm64.jar
• opencv-4.0.1-1.5-ios-x86_64.jar
• opencv-4.0.1-1.5-linux-armhf.jar
• opencv-4.0.1-1.5-linux-ppc64le.jar
• opencv-4.0.1-1.5-linux-x86.jar
• opencv-4.0.1-1.5-linux-x86_64.jar
• opencv-4.0.1-1.5-macosx-x86_64.jar
• opencv-4.0.1-1.5-windows-x86.jar
• opencv-4.0.1-1.5-windows-x86_64.jar

All of them conform to one of these patterns:

• ${artifactId}-${version}-${osfamily}-${arch}.jar
• ${artifactId}-${version}-natives-${osfamily}-${arch}.jar

Where arch is one of:

• aarch64
• amd64
• arm
• arm64
• armhf
• armv6
• armv6hf
• i586
• ppc64le
• universal (not really—it's just x86_64 + arm64 together, right?)
• x86
• x86_64

The native-lib-loader scheme is different in that it uses 32 as a shorthand for x86/i586, and 64 as a shorthand for amd64/x86_64.

Maybe we should stop using the bits variable in the default scijava.natives.classifier declaration, and instead use natives-${scijava.platform.family.medium}_${scijava.platform.arch.short}? And scijava.platform.arch.short = os.arch except in the cases of the shorthands described above?

Sorry for the delay @ctrueden but yes I agree with you, the current bits variable is only going to give us that 32/64 which most of the time is not helpful as we still have no idea of whether it's x86_64/arm64 etc.

to add to that as indicated here flimlib/flimlib#60 (comment) at present bits doesn't even expand to anything on M1

I made a new issue #30 to continue planning, discussion, implementation of the needed improvements to the scijava.platform properties.

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/migrating-from-fiji-clij-to-napari-pyclesperanto/54985/73

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/cmake-fails-to-build-on-m1-mac-apple-silicon/61964/8

With 81f583d, pom-scijava-base now has better support for Mac M1 (arm64) architecture, including correct naming for javacpp- and jogl-based native classifier artifacts.