Repository navigation
Force scalaVersion when using scala scalafmt extension #1273
Description
Activity
I would expect that the scala version wouldn't depend on
-PscalaVersion, and here is why:this.jarState = JarState.from(mavenCoordinate + version, provisioner); The scalaFmt classpath is resolved in an isolated way, as shown above. The
provisionerfield is provided as belowspotless/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/GradleProvisioner.java
Lines 111 to 136 in ce98b68
private static Provisioner forConfigurationContainer(Project project, ConfigurationContainer configurations, DependencyHandler dependencies) { return (withTransitives, mavenCoords) -> { try { Configuration config = configurations.create("spotless" + new Request(withTransitives, mavenCoords).hashCode()); mavenCoords.stream() .map(dependencies::create) .forEach(config.getDependencies()::add); config.setDescription(mavenCoords.toString()); config.setTransitive(withTransitives); config.attributes(attr -> { attr.attribute(Bundling.BUNDLING_ATTRIBUTE, project.getObjects().named(Bundling.class, Bundling.EXTERNAL)); }); return config.resolve(); } catch (Exception e) { String projName = project.getPath().substring(1).replace(':', '/'); if (!projName.isEmpty()) { projName = projName + "/"; } throw new GradleException(String.format( "You need to add a repository containing the '%s' artifact in '%sbuild.gradle'.%n" + "E.g.: 'repositories { mavenCentral() }'", mavenCoords, projName), e); } }; } I don't know how Gradle handles Scala, but I suppose that the scala plugin might modify the dependency resolution process somehow. Happy to take a PR that fixes this, but it seems like this should probably be pretty easy to workaround by disabling the spotless tasks on non-friendly Scala versions.
I don't know how Gradle handles Scala, but I suppose that the scala plugin might modify the dependency resolution process somehow.
Scalafmt is a scala project so it requires scala to be on the classpath to run so yes I do believe the gradle Scala plugin is actually providing that version of Scala on the classpath which is then picked up by spotless.
Happy to take a PR that fixes this, but it seems like this should probably be pretty easy to workaround by disabling the spotless tasks on non-friendly Scala versions.
Indeed I think as a workaround this should be fine and since it appears that scalafmt is using an isolated class it should be possible to provide an alternate Scala version, I will have a look at this tomorrow.
I created an issue at gradle (see gradle/gradle#21502) to also see if they have any ideas but I suspect to solve this issue we will need to use an isolated classloader due to the fact that we are going to have different scala versions in the same JVM instance.
There might also be an argument that in general any of the spotless integrations should run in an isolated classloader since most of the formatters/linters are cli "like" tools that work on context free grammers (i.e. they just parse source files)
@nedtwigg So I did some research on this issue (see gradle/gradle#21502 (comment)) and I better understand the problem/solutions.
So the essentially problem is that spotless-scala hardcodes the Scala version (see https://github.com/diffplug/spotless/blob/0fd20bb80c6c426d20e0a3157c3c2b89317032da/lib/src/main/java/com/diffplug/spotless/scala/ScalaFmtStep.java) depending on which scalafmt version you configure. Due to spotless not knowing anything about gradle-scala plugin, this causes spotless-scala to pick Scala 2.13.x even if you configure the Scala version via gradle-scala to be 2.12.x and configuring gradle-scala to Scala 2.12 will override the Scala version globally in gradle causing gradle to load the scala lib/runtime for that version.
This leaves us with 2 solutions
- Modify spotless to detect if gradle-scala is loaded as a plugin, and if is loaded pick the scala version that is configured by gradle-scala rather than the current hardcoding behaviour. I am not that familiar with gradle plugins but I assume that gradle does provide a mechanism to detect if you have a specific plugin loaded and also introspect the configuration for that plugin (in this case the configured scala version). Note that scalafmt 3.x is cross compiled for Scala 2.12 and 2.13 so we shouldn't have issue resolving multiple Scala versions as long as someone is not running something ancient (i.e. something pre Scala 2.12.x).
- Use a classloader based approach where we completely isolate the scalafmt plugin in its own classloader that is separate from gradle-scala. Theoretically speaking this is the "best" solution because it means that spotless-scala will work irrespective of anything else and scalafmt does have its own API which already runs the formatter in its own classloader (this is how sbt scala's main build tool solves the problem). There are a couple of problems here however, one is that it may not even work because gradle-scala plugin itself may not even load Scala in its own classloader which if so would make this solution impossible, quote
I think it's not possible currently to have two different Scala versions on the classpath when using Gradle. Gradle Scala plugin needs a specific version and I don't think it's isolated behind a separate classloader. As long as both spotless and Scala plugin use the same classpath it will not work (unless I am mistaken and it actually does separate classloaders 🤔 )
Even if this is not the case I suspect it would also be a more complex solution that would take longer to implement correctly.
wdyt? For me I have a preference of detecting if gradle-scala is being used and just picking the scala version from that. Also willing to do a PR however I am not that familiar with gradle so if there is some quick reference/sample code and/or documentation on how to dynamically detect plugins that would be great.
A few things - for one thing, if we wanted to remove the reflection, that is possible
Also we are using an isolated classloader. We grab the jar files for ScalaFmt here
this.jarState = JarState.from(mavenCoordinate + version, provisioner); and then use that isolated classloader here
ClassLoader classLoader = jarState.getClassLoader(); detect if gradle-scala is loaded as a plugin
You can do that like so:
spotless/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/ScalaExtension.java
Lines 73 to 76 in b535dce
JavaPluginConvention javaPlugin = getProject().getConvention().findPlugin(JavaPluginConvention.class); if (javaPlugin == null) { throw new GradleException("You must either specify 'target' manually or apply the 'scala' plugin."); } @nedtwigg I just made a PR that converts the scalafmt integration to use compile time source sets (see #1283). I still need to test this with Kafka, but if the issue due to reflection this should be enough to solve it. If not I will do a future PR which will try to detect a currently configured Scala version using the classpath.
I want to test if this PR solves my problem by publishing
spotlesslocally using./gradlew publishToMavenLocalhowever I am getting problems with signing, i.e.Execution failed for task ':plugin-gradle:signPluginMavenPublication'. > Cannot perform signing task ':plugin-gradle:signPluginMavenPublication' because it has no configured signatoryDo you have any ideas how to temporarily disable signing just for local publish?
Add
-x signPluginMavenPublicationto the gradle command line.Thanks for the tip, I ended up just disabling the
signingplugin which allowed me to publish locally however I am getting problems resolving amavenLocal()repository due to conflicts between with the defaultgradlePluginPortal()(a clash due to the same groupId with artifactory + snapshots is causing artifactory to return a 409).In any case I have already implemented what I described at #1283 (comment), I just haven't committed the changes yet.
Thanks to @mdedetrich for the new
majorScalaVersionparameter, published inplugin-gradle 6.10.0andplugin-maven 2.25.0.Thanks, I can confirm that the new release has also solved the underlying problem I had wrt Scala binary version.
Related to #579
Currently I am having an issue in upgrading Kafka's scala's spotless plugin to the latest version of scalafmt (3.5.8), you can see the PR here apache/kafka#12475 (comment). While the spotless scala plugin works fine when running with the default Scala version that is 2.13.8, when happen to build Kafka with a different scala version by using the
-PscalaVersionflag it causes spotless to fail, i.e.Runs fine without any problems, but if you do
Then you get the following stack trace
The reason behind the stacktrace is the fact that the latest version of Scalafmt (3.5.8) is compiled against Scala 2.13.x but the usage of
-PscalaVersion=2.12happens to also override the Scala version used by Scalafmt. Note that the reason why we specify the version ofScala using
-PscalaVersion=2.12is because of building and then releasing Kafka against multiple Scala versions.Ultimately the point is that whatever version Scalafmt happens to run is irrelevant to what Scala version you are using to build your project and in this specific case we would ideally want to force a scalaVersion that scalafmt with regardless of the
-PscalaVersion=2.12flag, i.e. something likeRelevant details:
Spotless version: 6.9.0
Operating system: MacOS Montery 12.5 M1
Spotless Configuration