Binding: Vision

@jonpryor

The current binding infrastructure is "janky":

"Somehow obtain" a .aar or .jar file to bind.
Add it to a Binding project, and set the appropriate build action, e.g. @(LibraryProjectZip) or @(EmbeddedJar).
Build it.

In practice, (1) and (3) can be incredibly annoying. (At least for @jonpryor, it's not always obvious where to obtain the files referenced by a Maven artifact, and (3) is full of peril.)

(3) will remain "out of scope" for this vision; fixing (3) involves ensuring that generator can always emit compilable output, and that we expand the Java constructs that generator supports. We should absolutely do this, but addressing these deficits involves finding the "less-than-ideal" constructs, filing issues for them, and fixing them.

Which leaves (1) and (2): can anything be done there?

Artifact Retrieval & Binding

Instead of a .aar or .jar file as being the "primary starting point", can we instead have a Maven artifact ID be the primary point?

Then, given a set of Maven artifact IDs:

Specify the Maven IDs.
Obtain the entire dependency graph for the maven artifacts.
Find existing bindings for everything within (1).
For any dependencies not already bound, bind them.

Specify the Maven IDs

How should the Maven IDs be specified? What's the "user interface"?

Obtain Maven Dependency Graph

I believe @moljac has code to do this, as part of Binderator: https://github.com/xamarin/XamarinComponents/tree/master/Util/Xamarin.AndroidBinderator

Find existing bindings

It appears that NuGet.org allows searching arbitrary package metadata. If we ensure that all new NuGet packages contain NuGet.org-searchable package metadata containing the Maven artifact ID & version, then it should be possible to find existing bindings.

@moljac is working on this.

Bind unbound packages

For any maven artifacts in the dependency graph which haven't been bound, presumably including the "root" dependencies, we should bind them.

Proposal: Item Group?

We can imagine using an MSBuild item group to specify the "root" maven IDs, either directly:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

or by using a file (file format? Gradle? JSON? XML?) to contain the information.

Implicit in this idea is that these are like @(PackageReference) item group: the containing .csproj can have as many of them as desired, which in turn means that the .csproj could emit "many" output assemblies (dozens?), one per bound library as well as the "intentioned" output assembly.

Additionally, how would @(TransformFile) (e.g. metadata.xml) work in such an environment? All @(TransformFile)s apply to all unbound artifacts?

Proposal: `dotnet new`?

Instead of using an item group, add a new tool, e.g.

dotnet new maven-binding --group-id androidx.car --artifact-id car --version 1.0.0-alpha7

This command would in turn emit a solution, with one .csproj project per unbound artifact. This "answers" the @(TransformFile) question: each project uses has its own set of @(TransformFile)s.

Less well defined is this: how does one update an existing "binding solution"? Changing the root artifact version will "ripple" across the entire solution; is it possible to update such a directory tree in-place? Or would we need to re-invoke the dotnet new maven-binding command into a new directory, and require the user manually copy over any required @(TransformFile)s & partial classes and such?

is it possible to update such a directory tree in-place?

When working on #5348 I found you can do dotnet new maven-binding --force to write on top of existing directories and files. It does not delete any existing files when using --force.

It sounds like this proposal is basically doing 2 things for the user:

Downloading the binary from Maven
Adding NuGet or Project dependencies automatically based on Maven metadata

#1 probably isn't too interesting. I don't think it used to be this way, but now you can just search on Maven and it provides an easy link to download the artifact: example.

We could probably even automate this without much effort in today's projects if we wanted to and resolve it at compile time:

Change:

<InputJar Include="okhttp-4.9.0.jar" />

to something like:

<MavenInputJar Group="com.squareup.okhttp3" Id="okhttp" Version="4.9.0" />

#2 is definitely more interesting. I'm not sure how much effort it normally takes.

Additional Considerations to Flesh Out

NuGet Metadata

From my reading of that link, I don't think NuGet supports searching arbitrary metadata. It seems to only support searching the official NuGet supported metadata. Thus we would need to decide which of those fields we would use.

Additions considerations:

How do we handle collisions? ie: More than one package says they are the package for com.squareup.okhttp3.okhttp.
Is this a security risk? Does it matter if someone publishes a malicious package claiming to be androidx.core on NuGet?

Visual Studio UX

How extensible is the built-in VS template UI system? Can we prompt the user to enter Maven information that we can use? Is the template system robust enough that it can run our custom logic to query Maven and resolve dependencies in order to generate projects?

If we can only do this outside of VS, do we think users will find and use it?

Updates

Once a package has been bound and a new release is posted to Maven, what does the update process look like for updating our binding?

Cases to consider:

Update the bound package version
New version has a new dependency that is on NuGet
New version has a new dependency that is not on NuGet
New version removed a NuGet dependency
New version removed a not-NuGet dependency

Publishing to NuGet

We need to understand what the target user is going to do with the binding: consume it locally or publish to NuGet.

If publishing to NuGet then there are a few additional issues like:

Specifying the NuGet package version
How do we convert project dependencies we created into NuGet dependencies once published?

It sounds like this proposal is basically doing 2 things for the user:

Downloading the binary from Maven

Trivial.

Adding NuGet or Project dependencies automatically based on Maven metadata

Not only Maven. What if we have bindings? maven gives a sheet about nuget.org

From my reading of that link, I don't think NuGet supports searching arbitrary metadata.
It seems to only support searching the official NuGet supported metadata.
Thus we would need to decide which of those fields we would use.

I am testing client API for search

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.NuGet/NuGetClient.cs#L36-L73

NOTE:

                                                    new SearchFilterType
                                                    {                                                       
                                                    }

I did not play with that more than 15 minutes.

How do we handle collisions? ie: More than one package says they are the package for com.squareup.okhttp3.okhttp.

dunno. Trying to have something - then I can make new steps...

Cases to consider:

Update the bound package version
New version has a new dependency that is on NuGet
New version has a new dependency that is not on NuGet
New version removed a NuGet dependency
New version removed a not-NuGet dependency

Writing xummary and pseudo-code. Coming soon

@moljac

Obtain Maven Dependency Graph
I believe @moljac has code to do this, as part of Binderator:
https://github.com/xamarin/XamarinComponents/tree/master/Util/Xamarin.AndroidBinderator

Nope. Side project. Will be easily embeddable. Or code copied with whatever changes needed (namespace changes etc)

Proposal: Item Group?
We can imagine using an MSBuild item group to specify the "root" maven IDs, either directly:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

MavenArtifact would not normaly appear in .NET library or project, but Xamarin.Android. Thus we could add some convetion-based rules... Like checking for serialized data files (json, xml) with artifacts to bind...

If the ItemGroup is defined we could:

check for serialized files and merge lists
or
ignore files

Why working with files?
It would be easier to have such files as "interfaces" for design and testing.

Now more details:

We have 2 sources od ultimate divine truth (priority list):

nuget.org
maven repos (currently I work only with maven.google.com)

Pick this:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

What I am trying to implement right now:

query nuget.org if there is already something like "androidx car car" on nuget.org. Grab versions and metadata
https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.NuGet/NuGetClient.cs#L43
If there is NO such nuget package or the version is GREATER-THAN Version="1.0.0-alpha7" put in the List TODOs_ToBind

Check dependencies from maven POM

foreach (Artifact a in a.POM.Dependencies.Leaves()) // Leaves is GraphTheoretical term for flattening
{
    TODOs_ToBind = TODOs_ToBind.Concat
                                (
                                    ! a.CheckNugetOrgPackages()     // everything not on nuget.org must be bound 
                                                                    // .. or other BuildActions 
                                );
}

foreach (Artifact a in TODOs_ToBind)
{
    Process.Run("dotnet new maven-binding {a}");
}
'''

Artifact a  = new Artifact().POM;

POM does not have stable name (yet) it is basically Maven 4.0.0 XSD schema transformed into C# classes

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/docs/maven-4.0.0.xsd

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.Maven/Models.GeneratedFromXSD.Original/maven-4.0.0.cs

I used XML to from AndroidX and GPS-FB to generate C#, - BUT there were some fields missing. So last week I generated C# from maven XSD

My plan is 20201202 for next 2 weeks:

create minimal API for step 3 in Binding: Vision #5352 (comment)
pack/publish some HolisticWare packages with API
create Cake scripts for testing that can be copied ANYWHERE (C# libs for MSBuild tasks or dotnet tools)

@Redth

An additional consideration from @Redth:

Maven is like NuGet, there can be many feeds, some of which may require authentication. So we may need to support specifying a URL, and possibly a username/password as well.

We should really back up a step here and sit down to discuss a few things:

What are the different scenarios in which customers care about bindings
What are the challenges in those scenarios
What can we do to help with those challenges.

What are the different scenarios in which customers care about bindings

In very few cases they do. Usually expect to be done by Microsoft or some partner. Basically wishes to have some SDK bindings are submitted as issues in (AndroidX/AndroidSupport, GPS-FB and XamarinComponents). Some are submitted as internal mails and very few (for me personally) over social media.

What are the challenges in those scenarios

Bindings knowledge and skills which influence productivity where most users give up.
Bindings process
2.1 find/download artifact,
2.2 analyze artifact (dependencies and nuget packages)
Test bindings (samples or unit tests)

Challenges I personally have:

Does bindings package exist for given artifact?
Is update necessary?
What are details on NuGet.org?
- Dependencies (transitive)
Where to get artifact[s] (download) and info about it?
I use following repositories (in this order)
https://maven.google.com/web/index.html
https://mvnrepository.com/
https://search.maven.org/
What are details (info about) artifact from Maven repo?
- Dependencies (transitive)
  - Is something to be bound in transitive dependency chain?
    For published bindings on NuGet.org this is not necessary
    - How to handle transitive dependencies?
      dotnet new?
      BuildAction?

What can we do to help with those challenges.

Improving productivity will most likely increase abstraction (add one or more levels of APIs) that might drive some users away from bindings.

Needed APIs:

NuGet querying for info about bindings

After few days of more intensive work with NuGet client API the conclusion is that there is much more work
than expected. NuGet.org does not have perfect search, so client API is limited too.
Few problems:
- Search does not retrieve DependencySets, so filtering by TargetFramework is more complicated then I though
- Search API input is string as keyword retrieving tons of results.
  Current solution is to provide API with Predicate or planned PredicateBuilder for custom search
  and to extend it with Metadata query which retrieves Package DependencySetsProperly
Maven APIs (extended - more than in MavenNET)
API that uses data from NuGet API and Maven API and prepares bindings

NOTE: More details to come.

Maven is like NuGet, there can be many feeds, some of which may require authentication. So we may need to support specifying a URL, and possibly a username/password as well.

I work with Google's maven right now, but I do plan to add others (mavenCentral and jcenter/Bintray).

I would not bother with private repos or repos that need authentication. There must be some reason artifacts are not public. We could add that later.

Maven API (extended)

20201208

Initial data for current version of binderator:

      {
        "groupId": "androidx.car",
        "artifactId": "car",
        "version": "1.0.0-alpha7",
        "nugetVersion": "1.0.0.3-alpha7",
        "nugetId": "Xamarin.AndroidX.Car.Car",
        "dependencyOnly": false
      },

https://github.com/xamarin/AndroidX/blob/master/config.json#L172-L179

NOTE:

For AndroidX and GPS-FB there is almost 200 artifacts to bind with growing number.
So, I want to free my time so I can do other stuff including these improvements.
Thus starting with binderator's config.json

Because it is manual process (currently) order of json data is important for me, because I can easily check/verify
versions and names.

Artifact Versions

So with 20201208 there is extended info with versions (available for artifact):

{
  "groupId": "androidx.car",
  "artifactId": "car",
  "version": null,
  "idFullyQualified": "androidx.car.car",
  "versionTextual": "1.0.0-alpha5",
  "versions": null,
  "versionsTextual": [
    "1.0.0-alpha7",
    "1.0.0-alpha5",
    "1.0.0-alpha4",
    "1.0.0-alpha3",
    "1.0.0-alpha1"
  ],
  "projectObjectModelTextual": null,
  "projectObjectModel": null,
  "dependencies": null
}

This will provide opportunity to verify version given for bindings ("1.0.0-alpha6" would fail) and to
make plan for future bindings or even automate (generate next config, based on current)

Cache

WIP - deserializing locally dumped Maven repo data - if not grab it.

ProjectObjectModel POM

Currently POM is grabbed and deserialized.

WIP on extracting important data like dependecies:

{
  "groupId": "androidx.car",
  "artifactId": "car",
  "version": null,
  "idFullyQualified": "androidx.car.car",
  "versionTextual": "1.0.0-alpha5",
  "versions": null,
  "versionsTextual": null,
  "projectObjectModelTextual": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\" xmlns=\"http://maven.apache.org/POM/4.0.0\"\n    xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n  <modelVersion>4.0.0</modelVersion>\n  <groupId>androidx.car</groupId>\n  <artifactId>car</artifactId>\n  <version>1.0.0-alpha5</version>\n  <packaging>aar</packaging>\n  <name>Android Car Support UI</name>\n  <description>Android Car Support UI</description>\n  <url>http://developer.android.com/tools/extras/support-library.html</url>\n  <inceptionYear>2017</inceptionYear>\n  <licenses>\n    <license>\n      <name>The Apache Software License, Version 2.0</name>\n      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>\n      <distribution>repo</distribution>\n    </license>\n  </licenses>\n  <developers>\n    <developer>\n      <name>The Android Open Source Project</name>\n    </developer>\n  </developers>\n  <scm>\n    <connection>scm:git:https://android.googlesource.com/platform/frameworks/support</connection>\n    <url>http://source.android.com</url>\n  </scm>\n  <dependencies>\n    <dependency>\n      <groupId>com.google.android.material</groupId>\n      <artifactId>material</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n      <exclusions>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.appcompat</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.annotation</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.transition</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.recyclerview</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.legacy</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.fragment</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.cardview</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.core</groupId>\n        </exclusion>\n      </exclusions>\n    </dependency>\n    <dependency>\n      <groupId>androidx.appcompat</groupId>\n      <artifactId>appcompat</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.cardview</groupId>\n      <artifactId>cardview</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.annotation</groupId>\n      <artifactId>annotation</artifactId>\n      <version>1.0.0-rc01</version>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.legacy</groupId>\n      <artifactId>legacy-support-v4</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.recyclerview</groupId>\n      <artifactId>recyclerview</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.gridlayout</groupId>\n      <artifactId>gridlayout</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.preference</groupId>\n      <artifactId>preference</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.constraintlayout</groupId>\n      <artifactId>constraintlayout</artifactId>\n      <version>1.1.0</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n  </dependencies>\n</project>\n",
  "projectObjectModel": {
    "modelVersion": "4.0.0",
    "parent": null,
    "packaging": "aar",
    "name": "Android Car Support UI",
    "description": "Android Car Support UI",
    "url": "http://developer.android.com/tools/extras/support-library.html",
    "inceptionYear": "2017",
    "organization": null,
    "licenses": [
      {
        "name": "The Apache Software License, Version 2.0",
        "url": "http://www.apache.org/licenses/LICENSE-2.0.txt",
        "distribution": "repo",
        "comments": null
      }
    ],
    "developers": [
      {
        "id": null,
        "name": "The Android Open Source Project",
        "email": null,
        "url": null,
        "organization": null,
        "organizationUrl": null,
        "roles": null,
        "timezone": null,
        "properties": null
      }
    ],
    "contributors": null,
    "mailingLists": null,
    "prerequisites": null,
    "modules": null,
    "scm": {
      "connection": "scm:git:https://android.googlesource.com/platform/frameworks/support",
      "developerConnection": null,
      "tag": "HEAD",
      "url": "http://source.android.com",
      "childscmconnectioninheritappendpath": null,
      "childscmdeveloperConnectioninheritappendpath": null,
      "childscmurlinheritappendpath": null
    },
    "issueManagement": null,
    "ciManagement": null,
    "distributionManagement": null,
    "properties": null,
    "dependencyManagement": null,
    "repositories": null,
    "pluginRepositories": null,
    "build": null,
    "reports": null,
    "reporting": null,
    "profiles": null,
    "childprojecturlinheritappendpath": null,
    "groupId": "androidx.car",
    "artifactId": "car",
    "version": "1.0.0-alpha5",
    "dependencies": [
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": [
          {
            "groupId": "androidx.appcompat",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.annotation",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.transition",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.recyclerview",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.legacy",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.fragment",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.cardview",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.core",
            "artifactId": "*"
          }
        ],
        "optional": null,
        "groupId": "com.google.android.material",
        "artifactId": "material",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.appcompat",
        "artifactId": "appcompat",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.cardview",
        "artifactId": "cardview",
        "version": "1.0.0-rc01"
      },
      {
        "type": "jar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.annotation",
        "artifactId": "annotation",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.legacy",
        "artifactId": "legacy-support-v4",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.recyclerview",
        "artifactId": "recyclerview",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.gridlayout",
        "artifactId": "gridlayout",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.preference",
        "artifactId": "preference",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.constraintlayout",
        "artifactId": "constraintlayout",
        "version": "1.1.0"
      }
    ]
  },
  "dependencies": null
}

Extras

order of serialized data

Goal is to have order of serialized data as close as possible to binderator's config files,
so that manual work can be continued.
cache
serialization with buddy (metadata classes)
textual data in debug mode

NuGet API

20201208

Currently there are 3 Nuget Client API use cases:

Search
GetMetadata
GetVersions

Search is very limited regarding filtering and customizations and searches using Contains() API.
So, API with Predicate had to be added and

Furthermore Search does not return DependecySets to filter out based on TargetFramework, so for each
nuget GetMetadataAsync(...) must be called. I open an issue in Nuget Client API repo. The idea was not
to surface NuGetClient classes, but for now I had to.

Search

So, the closest API in few days that returns the same results (3 results for "androidx+car" search ) is:

            ArtifactBindingNuget abnd = new ArtifactBindingNuget("androidx.car", "car")
            {
                NuGetId = "Xamarin.AndroidX.Car.Car"
            };

            IEnumerable<global::NuGet.Protocol.Core.Types.IPackageSearchMetadata> result = null;

            result = abnd.SearchPackagesByKeywordWithFilterAsync
                            (
                                abnd.NuGetId,
                                // null,
                                new global::NuGet.Protocol.Core.Types.SearchFilter
                                                                        (
                                                                            includePrerelease: true
                                                                        ),
                                skip: 0,
                                take: 100,
                                // default (for null) predicates:
                                // 
                                // custom predicate:
                                psm =>
                                {
                                    return
                                    (
                                        (
                                            psm.Title.ToLower().Contains("androidx")
                                            &&
                                            psm.Title.ToLower().Contains("car")
                                        )
                                        &&
                                        (
                                            psm.Description.ToLower().Contains("car")
                                            ||
                                            psm.Description.ToLower().Contains("androidx.car")
                                        )
                                    );
                                }
                            )
                            .Result;

And it returns serialized:

{
  "nuGetId": "Xamarin.AndroidX.Car.Car",
  "nuGetPackagesSearchResults": [
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - car",
      "downloadCount": 539,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.car.car/1.0.0.3-alpha7/icon",
      "identity": {
        "id": "Xamarin.AndroidX.Car.Car",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 3,
          "isSemVer2": false,
          "originalVersion": "1.0.0.3-alpha7",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [
            "alpha7"
          ],
          "release": "alpha7",
          "isPrerelease": true,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 3,
            "majorRevision": 0,
            "minorRevision": 3
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.Car.Car/1.0.0.3-alpha7/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - car",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, car",
      "title": "Xamarin AndroidX - car",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    },
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - cardview",
      "downloadCount": 1080465,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.cardview/1.0.0.5/icon",
      "identity": {
        "id": "Xamarin.AndroidX.CardView",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 5,
          "isSemVer2": false,
          "originalVersion": "1.0.0.5",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [],
          "release": "",
          "isPrerelease": false,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 5,
            "majorRevision": 0,
            "minorRevision": 5
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.CardView/1.0.0.5/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - cardview",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, cardview",
      "title": "Xamarin AndroidX - cardview",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    },
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - car-cluster",
      "downloadCount": 549,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.car.cluster/1.0.0.3-alpha5/icon",
      "identity": {
        "id": "Xamarin.AndroidX.Car.Cluster",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 3,
          "isSemVer2": false,
          "originalVersion": "1.0.0.3-alpha5",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [
            "alpha5"
          ],
          "release": "alpha5",
          "isPrerelease": true,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 3,
            "majorRevision": 0,
            "minorRevision": 3
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.Car.Cluster/1.0.0.3-alpha5/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - car-cluster",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, car-cluster",
      "title": "Xamarin AndroidX - car-cluster",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    }
  ]
}

Extras

order of serialized data

Goal is to have order of serialized data as close as possible to binderator's config files,
so that manual work can be continued.
cache
custom predicates (as static members) for filtering
- TargetFramework
serialization with buddy (metadata classes)
textual data in debug mode

I did a spike on a lot of this some time ago: https://github.com/Redth/Xamarin.Binding.Helpers

The idea was to use a .gradle file to specify the dependencies for the binding and let gradle/maven resolve the dependency graph for you, since that's what it's good at. With a small plugin added to the .gradle file, I was able to serialize all of the dependency graph from gradle in a way that msbuild tasks could parse it out and use it to obtain the dependency tree info (maven artifact group, id, version, binaries, etc). This could all be completely automated such that a gradle config is created for the top level maven artifacts specified by the developer and gradle can be downloaded and invoked on behalf of the user to get this info. The bonus here is things like maven feed authentication do not need re-inventing or engineering.

Once the dependency graph was known, all of the actual aar/jar binaries could be downloaded and included as appropriate items in the binding project. However, as mentioned in this issue several times, there's already lots of nuget packages out there that ship these artifacts in binding projects and would conflict during builds. Often these packages are desirable instead of just bundling the artifacts since their binding c# api surface may be required.

I did some work in the same repo around matching up NuGet packages. The idea of having metadata in nuget packages to match up their maven artifacts is great.... except... we'll likely never be able to get the entire ecosystem updated to this model. If that's an assumption then this mechanism would only be reliable for packages we control, or otherwise well known community packages. If we assume that, we can pretty much identify that set of packages already and map out some rules for the artifacts that packages map.

This is exactly what I did here: https://github.com/Redth/Xamarin.Binding.Helpers/blob/main/Xamarin.Binding.Helpers/NuGetResolvers/AndroidXMavenNugetResolver.cs

There are others for Firebase, GPS, and other well known package: https://github.com/Redth/Xamarin.Binding.Helpers/blob/main/Xamarin.Binding.Helpers/NuGetResolvers/KnownMavenNugetResolver.cs#L12-L98

It's really not that bad of a set of rules/mappings to have/maintain perhaps in addition to new packages being able to publish maven artifact related metadata (and fallback to these mappings).

The idea was that these rules would apply, and/or you could specify explicit mappings yourself if you know they exist, like this:

<ItemGroup>
  <MavenNuGetMapping MavenGroupId="com.google.zxing" MavenArtifactId="core" MavenVersion="3.3.3" NuGetPackageId="Xamarin.Google.ZXing.Core" NuGetVersion="3.3.3" /></ItemGroup>

So, for this whole library you would basically reference and android studio / gradle project in your xamarin binding project like this:

<ItemGroup>
  <AndroidStudioProject Include="/path/to/project" Module="mylibrary" GenerateBinding="True" />
</ItemGroup>

This would even go build the native project for you and include the module's binary (aar) as a so that it would generate bindings (or not if you disabled that option). It would link in the entire dependency tree of maven artifacts based on the android project's declared compile dependencies, but it would substitute those for NuGet packages according to the resolvers or explicit mapping override info you specified.

This isn't a perfect approach by any means, and I could see taking it a step further to not require the android studio project at all, but this is a good start to look at.
Generally you wouldn't need to do this

It looks like gradlew -I INIT-SCRIPT can be specified multiple times: https://github.com/gradle/gradle/blob/edb2cc76c1d398633843303a42ad683db0fe8393/subprojects/core/src/main/java/org/gradle/initialization/StartParameterBuildOptions.java#L238-L251

Consequently, it should be possible to distribute a .gradle file with "something" (Android SDK, NuGet package) and pass this file to gradlew when invoking it. (Perhaps via a @(AndroidGradleInitScript) item group?)

Assuming this works, it means we can reliably add a Gradle task to retrieve the artifacts of the Gradle build, so that we can automagically determine paths to built artifacts + dependencies.

I've been giving this some thought recently, and maybe we should do less automatically for the user (since that's where the dragons live), and focus first on helping the user ensure they are creating correct bindings.

TL:DR

We will initially focus on tackling two pain points of binding from Maven:

Acquiring the .jar/.aar and the related .pom from Maven
Using the .pom to verify that required Java dependencies are being fulfilled

Moved proposal details to #4528.

We will initially focus on tackling two pain points of binding from Maven:

Acquiring the .jar/.aar and the related .pom from Maven

Using the .pom to verify that required Java dependencies are being fulfilled

These features have been implemented for .NET 9 in the following PRs:

I think the rest of this issue is conjectural and not really actionable. Any additional steps we want to consider taking should probably be proposed and fleshed out in new issue(s).

Navigation Menu

Repository navigation

Binding: Vision #5352

Artifact Retrieval & Binding

Specify the Maven IDs

Obtain Maven Dependency Graph

Find existing bindings

Bind unbound packages

Proposal: Item Group?

Proposal: `dotnet new`?

Additional Considerations to Flesh Out

NuGet Metadata

Visual Studio UX

Updates

Publishing to NuGet

Maven API (extended)

Artifact Versions

Cache

ProjectObjectModel POM

Extras

NuGet API

Extras

TL:DR

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Binding: Vision #5352

Description

Artifact Retrieval & Binding

Specify the Maven IDs

Obtain Maven Dependency Graph

Find existing bindings

Bind unbound packages

Proposal: Item Group?

Proposal: dotnet new?

Activity

jonathanpeppers commented on Dec 1, 2020

jpobst commented on Dec 2, 2020

Additional Considerations to Flesh Out

NuGet Metadata

Visual Studio UX

Updates

Publishing to NuGet

moljac commented on Dec 2, 2020

moljac commented on Dec 2, 2020

moljac commented on Dec 2, 2020

moljac commented on Dec 2, 2020

jpobst commented on Dec 2, 2020

Redth commented on Dec 3, 2020

moljac commented on Dec 8, 2020

moljac commented on Dec 8, 2020

Maven API (extended)

Artifact Versions

Cache

ProjectObjectModel POM

Extras

moljac commented on Dec 8, 2020

NuGet API

Extras

Redth commented on Sep 14, 2021

jonpryor commented on Sep 12, 2022

jpobst commented on Sep 13, 2022

TL:DR

jpobst commented on Apr 17, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Proposal: `dotnet new`?