Binding Kotlin itself crashes generator --only-xml-adjuster

I am trying to bind Kotlin so that it can be consumed by Kotlin-based libraries, and the generator is not happy.

This is the library kotlin-stdlib-1.3.41.jar

I just create a new binding project and build. Because at this point I don't want any actual types, I added <remove-node path="/api/package" /> to the transforms.

The generator crashes with this error:
https://gist.github.com/mattleibow/92dff72582ec972b214283d8fe739a9c

The api.xml.class-parse file is this:
https://gist.github.com/mattleibow/34af078b7aca818dd0c5381e075896d1

I can reproduce the crash at this point in the build:

mono /Library/Frameworks/Xamarin.Android.framework/Versions/10.0.0.4/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe \
    api.xml.class-parse \
    --only-xml-adjuster \
    --xml-adjuster-output=api.xml \
    --csdir=src \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/Java.Interop.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v9.0/Mono.Android.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.Core.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.Net.Http.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/Facades/System.Runtime.dll \
    --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll \
    --api-level=28 \
    --global \
    --public

More info can be found at: #464

[Xamarin.Android.Tools.Bytecode] Support RuntimeVisibleAnnotations

I don't know what's going on. Time for some musings to collect later...

I thought that RuntimeVisibleAnnotations might be involved, so I added support for parsing RuntimeVisibleAnnotations and RuntimeInvisibleAnnotations blobs to class-parse: #467.

This might be overthinking it, though the commit message in #467 points out an important thought:

How/Why does kotlin/collections/AbstractCollection$toString$1''s invoke() method mention the type E, when class-parse-generated XML output doesn't mention E within a <typeParameters/> block?

<typeParameters/> in turn comes from the Signature annotation, which I should have looked at earlier. For comparison purposes, consider android.app.FragmentHostCallback. If we run class-parse --dump android/app/FragmentHostCallback.class, we get:

Attributes Count: 5
        0: Signature(<E:Ljava/lang/Object;>Landroid/app/FragmentContainer;)

The Signature annotation blob has a grammar for ClassSignature, which has FormalTypeParameters at the start of the string, between <...> brackets. This is what we see with android.app.FragmentHostCallback: a FormalTypeParameters specification of <E:Ljava/lang/Object;>, introducing the type parameter E which is constrained to java.lang.Object, followed by the signature of the super (base) class, android.app.FragmentContainer.

Now, what does kotlin/collections/AbstractCollection\$toString\$1.class have for it's Signature blob?

Attributes Count: 5
        0: InnerClasses(Count=1, InnerClass(InnerClass='kotlin/collections/AbstractCollection$toString$1', OuterClass='', InnerName='', InnerClassAccessFlags=Static, Final))
        1: Unknown[EnclosingMethod](4)
        2: Signature(Lkotlin/jvm/internal/Lambda;Lkotlin/jvm/functions/Function1<TE;Ljava/lang/CharSequence;>;)
        3: Unknown[SourceFile](2)
        4: RuntimeVisibleAnnotations(Annotation('Lkotlin/Metadata;', {mv: Array(Integer(1), Integer(1), Integer(15)), bv: Array(Integer(1), Integer(0), Integer(3)), k: Integer(3), d1: Array(Utf8("...")), d2: Array(Utf8("<anonymous>"), Utf8(""), Utf8("E"), Utf8("it"), Utf8("invoke"), Utf8("(Ljava/lang/Object;)Ljava/lang/CharSequence;"))}))

The Signature blob does not specify any type parameter information!

This leaves the question unanswered: where is E coming from? (I do see E in the RuntimeVisibleAnnotations for d2, but I don't want to begin to figure out what d1 and d2 are supposed to mean...)

Could the EnclosingMethod annotation be relevant?

[Xamarin.Android.Tools.Bytecode] Add EnclosingMethod, SourceFile

@jpobst

Thus, it appears to be due to attempted processing of generic types and their constraints.

It would be useful to know which exact type & method causes things to break. @jpobst mentioned a signature of ([Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; on kotlin/collections/ArraysKt___ArraysKt.class, but those methods look reasonable:

      <method
        abstract="false"
        deprecated="not deprecated"
        final="true"
        name="reduce"
        native="false"
        return="S"
        jni-return="TS;"
        static="true"
        synchronized="false"
        visibility="public"
        bridge="false"
        synthetic="false"
        jni-signature="([Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object;">
        <typeParameters>
          <typeParameter
            name="S"
            jni-classBound="Ljava/lang/Object;"
            classBound="java.lang.Object"
            interfaceBounds=""
            jni-interfaceBounds="" />
          <typeParameter
            name="T"
            jni-classBound=""
            classBound=""
            interfaceBounds="S"
            jni-interfaceBounds="TS;" />
        </typeParameters>
        <parameter
          name="$this$reduce"
          type="T[]"
          jni-type="[TT;" />
        <parameter
          name="operation"
          type="kotlin.jvm.functions.Function2&lt;? super S, ? super T, ? extends S&gt;"
          jni-type="Lkotlin/jvm/functions/Function2&lt;-TS;-TT;+TS;&gt;;" />
      </method>
      <method
        abstract="false"
        deprecated="not deprecated"
        final="true"
        name="reduceRight"
        native="false"
        return="S"
        jni-return="TS;"
        static="true"
        synchronized="false"
        visibility="public"
        bridge="false"
        synthetic="false"
        jni-signature="([Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object;">
        <typeParameters>
          <typeParameter
            name="S"
            jni-classBound="Ljava/lang/Object;"
            classBound="java.lang.Object"
            interfaceBounds=""
            jni-interfaceBounds="" />
          <typeParameter
            name="T"
            jni-classBound=""
            classBound=""
            interfaceBounds="S"
            jni-interfaceBounds="TS;" />
        </typeParameters>
        <parameter
          name="$this$reduceRight"
          type="T[]"
          jni-type="[TT;" />
        <parameter
          name="operation"
          type="kotlin.jvm.functions.Function2&lt;? super T, ? super S, ? extends S&gt;"
          jni-type="Lkotlin/jvm/functions/Function2&lt;-TT;-TS;+TS;&gt;;" />
      </method>

...although...perhaps it's having T have an interface bound on S which is causing the problem?

        <typeParameters>
          <typeParameter
            name="S"
            jni-classBound="Ljava/lang/Object;"
            classBound="java.lang.Object"
            interfaceBounds=""
            jni-interfaceBounds="" />
          <typeParameter
            name="T"
            jni-classBound=""
            classBound=""
            interfaceBounds="S"
            jni-interfaceBounds="TS;" />
        </typeParameters>

Which is, I believe, equivalent to:

<S, T implements S>
public static S reduceRight(T[] $this$reduceRight, Function2<? super T, ? super S, ? extends S> operation);

[Xamarin.Android.Tools.Bytecode] Add EnclosingMethod, SourceFile

[Xamarin.Android.Tools.Bytecode] Add EnclosingMethod, SourceFile (#469)

Revisiting #466 (comment)

How/Why does kotlin/collections/AbstractCollection$toString$1''s invoke() method mention the type E, when class-parse-generated XML output doesn't mention E within a <typeParameters/> block?

With PR #469 we now parse the EnclosingMethod attribute and export it from class-parse-generated XML. From it we see:

$ mono class-parse.exe kotlin/collections/AbstractCollection\$toString\$1.class
...
    <class
      abstract="false"
      deprecated="not deprecated"
      enclosing-method-jni-type="Lkotlin/collections/AbstractCollection;"
      enclosing-method-name="toString"
      enclosing-method-signature="()Ljava/lang/String;"
      jni-extends="Lkotlin/jvm/internal/Lambda;"
      extends="kotlin.jvm.internal.Lambda"
      extends-generic-aware="kotlin.jvm.internal.Lambda"
      final="true"
      name="AbstractCollection.toString.1"
      jni-signature="Lkotlin/collections/AbstractCollection$toString$1;"
      source-file-name="AbstractCollection.kt"
      static="true"
      visibility="">
...

The enclosing type is Lkotlin/collections/AbstractCollection;.

If we turn our attention to AbstractCollection, we see:

$ mono class-parse.exe kotlin/collections/AbstractCollection.class
...
    <class
      abstract="true"
      deprecated="not deprecated"
      jni-extends="Ljava/lang/Object;"
      extends="java.lang.Object"
      extends-generic-aware="java.lang.Object"
      final="false"
      name="AbstractCollection"
      jni-signature="Lkotlin/collections/AbstractCollection;"
      source-file-name="AbstractCollection.kt"
      static="false"
      visibility="public">
      <typeParameters>
        <typeParameter
          name="E"
          jni-classBound="Ljava/lang/Object;"
          classBound="java.lang.Object"
          interfaceBounds=""
          jni-interfaceBounds="" />

i.e. AbstractCollection does contain a declaration for the type parameter E, which is used/referenced by AbstractCollection$toString$1.

Thus, we can infer a possible fix: if a type has an enclosing-method-jni-type attribute, "lookup" the specified type (and optional method) to find any type parameters specified in the enclosing scope, and if necessary copy those <typeParameters/> into the resulting XML.

Alternatively, instead of having class-parse do that "copy <typeParameters/> to the referencing scope" logic, have generator perform that copy.

My concern over doing the copy at all is it alters the semantics of class-parse lib.jar vs. class-parse type.class: the former can perform "enclosing type" lookups; the latter can't. It may thus be preferable to have generator do the lookups, as it should have all necessary context.

[ApiXmlAdjuster] Guard against NRE when writing generic constraints. (#…

Bump to xamarin/Java.Interop/master@2716edfd

Bump to xamarin/Java.Interop/master@2716edfd (#3709)

Bump to xamarin/Java.Interop/d16-4@35b08baa (#3709)

[Xamarin.Android.Tools.Bytecode] Support Runtime[In]VisibleAnnotations

[Xamarin.Android.Tools.Bytecode] Runtime[In]VisibleAnnotations Parsing (

Fixed in #495.

Release status update

A new Release version of Xamarin.Android has now been published on Windows that includes the fix for this item. The fix is not yet published in a Release version on macOS. I will update this item again when a Release version is available on macOS that includes the fix.

Fix included in Xamarin.Android 10.1.0.30.

Fix included on Windows in Visual Studio 2019 version 16.4. To get the new version that includes the fix, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.

(Fix also included on macOS in Visual Studio 2019 for Mac version 8.4 Preview 2.1 and higher. To try the Preview version that includes the fix, check for the latest updates on the Preview updater channel.)

Release status update

A new Release version has now been published on macOS that includes the fix for this item.

Fix included in Xamarin.Android 10.1.1.0.

Fix included on macOS in Visual Studio 2019 for Mac version 8.4. To get the new version that includes the fix, check for the latest updates on the Stable updater channel.

(Fix also included on Windows in Visual Studio 2019 version 16.4 and higher. To get the new version that includes the fix, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.)

Navigation Menu

Repository navigation

Binding Kotlin itself crashes `generator --only-xml-adjuster` #466

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Binding Kotlin itself crashes generator --only-xml-adjuster #466

Description

Activity

mattleibow commented on Aug 13, 2019

jonpryor commented on Aug 13, 2019

jonpryor commented on Aug 14, 2019

jonpryor commented on Aug 19, 2019

jpobst commented on Oct 31, 2019

brendanzagaeski commented on Dec 4, 2019

brendanzagaeski commented on Jan 8, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Binding Kotlin itself crashes `generator --only-xml-adjuster` #466