Skip to content

[Microsoft.Android.Build.Tasks] Read NativeAOT typemap keys - #12825

Merged
simonrozsival merged 3 commits into
mainfrom
simonrozsival-typemap-nativeaot-object-adapter
Sep 26, 2026
Merged

simonrozsival merged 3 commits into
mainfrom
simonrozsival-typemap-nativeaot-object-adapter

Conversation

@simonrozsival

@simonrozsival simonrozsival commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Why

Layer 2 of 4 in native stack #12893, depending on #12824. This adds the NativeAOT representation adapter so typemap-based ProGuard rules can consume the keys retained in the final ILC native object rather than infer retention from a dependency graph.

Extraction contract

  • ExtractTypeMapKeysFromNativeAotObject reads final relocatable ELF objects using LLVM section/symbol metadata and common-fixup relocations to select the Java type-map groups by their EEType identities. It does not scan arbitrary strings or guess group membership from key appearance.
  • NativeAotTypeMapReader enumerates the selected NativeFormat groups, reusing the existing alias/array normalization and class-name validation helpers. Output is a deterministic union of canonical class keys; invalid required inputs fail with XA4327.
  • Add the frozen parser/adapter coverage, native-object fixture helpers, tool-path test metadata, and test-catalog instructions. All eight changed files match the frozen [Microsoft.Android.Build.Tasks] Generate typemap ProGuard rules #12821 snapshot exactly.

Tools and scope

Extraction requires an existing full path to NDK llvm-readobj with adjacent llvm-objdump; native test fixtures additionally need adjacent clang. Supply -p:_NativeAotLlvmReadObjPath=/path/to/ndk/bin/llvm-readobj for native-tool test cases.

This layer does not activate product targets, add assembly-resolution metadata, expand shared target integration tests, change R8 policy, enable NativeAOT trimming, or retire DGML. The existing DGML pipeline remains active; build activation and policy follow in layers 3 and 4.

Validation

Standalone trimmable typemap project built with .NET 11 preview 7 and -p:_AndroidTreatWarningsAsErrors=true, passing the existing NDK tool path. Targeted NativeAOT, linked-DLL, LLVM, ProGuard-generator, typemap-generator, and class-name coverage: 694 passed, 0 failed, 0 skipped.

NativeAotObjectIntegrationTests.cs contains shared tool/attribute helpers at this layer; real-object MSBuild integration test methods depend on the later shared target-test expansion and are not claimed as executed here.

Copilot AI lite review requested due to automatic review settings September 18, 2026 15:21
@simonrozsival simonrozsival added the r8-rules Typemap-derived ProGuard and R8 rules label Sep 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

It introduces a few correctness issues (tool-skip gating vs. required tools and potential partial-output writes) that should be addressed before the changes are safe to merge.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds NativeAOT object-file support for typemap key extraction in Microsoft.Android.Build.Tasks, enabling typemap-based ProGuard rule generation to consume keys retained in the final ILC-produced ELF object (via LLVM metadata/relocations) instead of inferring retention indirectly. This fits into the broader “typemap-derived ProGuard rules” pipeline by providing the NativeAOT adapter layer.

Changes:

  • Introduces ExtractTypeMapKeysFromNativeAotObject (MSBuild task) and NativeAotTypeMapReader (NativeFormat parser) to extract and validate canonical Java class keys from relocatable ELF objects.
  • Adds extensive unit tests and fixture generators for NativeFormat parsing and NativeAOT object metadata/relocation selection behavior.
  • Adds test-project metadata and test-catalog documentation for supplying NDK LLVM tool paths.
File Description
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​NativeAotObjectIntegrationTests.cs Adds shared tool-path resolution and skip attributes for NativeAOT object integration tests.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests.csproj Adds assembly metadata to pass _NativeAotLlvmReadObjPath into tests.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​NativeAotObjectTestFixture.Tool.cs Adds a clang-based generator for producing ELF .o fixtures containing typemap blobs and fixup tables.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​NativeAotObjectTestFixture.cs Adds NativeFormat blob/table/key encoding helpers for deterministic fixtures.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​ExtractTypeMapKeysFromNativeAotObjectTests.cs Adds comprehensive tests for metadata parsing, relocation-based group selection, normalization, and failure modes (XA4327).
src/​Microsoft.Android.Build.Tasks/​Utilities/​NativeAotTypeMapReader.cs Implements NativeFormat hashtable reader and key extraction/validation for selected Java groups.
src/​Microsoft.Android.Build.Tasks/​Tasks/​ExtractTypeMapKeysFromNativeAotObject.cs Implements the MSBuild task that reads ELF metadata + relocations via LLVM tools and writes canonical key output.
.github/​skills/​tests/​references/​test-catalog.md Documents additional native tool requirements and how to pass _NativeAotLlvmReadObjPath to run those tests.

@simonrozsival
simonrozsival added this pull request to stack #12830 September 18, 2026 19:14
@simonrozsival
simonrozsival force-pushed the simonrozsival-typemap-nativeaot-object-adapter branch from de949ac to 80c1aec Compare September 21, 2026 15:42
@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival
simonrozsival force-pushed the simonrozsival-typemap-nativeaot-object-adapter branch from 80c1aec to 1283bc7 Compare September 21, 2026 15:55
@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival
simonrozsival force-pushed the simonrozsival-typemap-nativeaot-object-adapter branch from 1283bc7 to f65f0d2 Compare September 22, 2026 22:02
@simonrozsival
simonrozsival removed this pull request from stack #12830 September 23, 2026 07:16
@simonrozsival
simonrozsival force-pushed the simonrozsival-typemap-nativeaot-object-adapter branch from f65f0d2 to c172dcb Compare September 23, 2026 07:17
@simonrozsival
simonrozsival added this pull request to stack #12893 September 23, 2026 07:45
Base automatically changed from simonrozsival-typemap-linked-dll-adapter to main September 24, 2026 14:58
Add the NativeAOT object adapter and NativeFormat reader from the frozen typemap ProGuard source snapshot. Select Java map groups through common-fixup relocations and enumerate canonical keys from the final ILC object.

Include parser and adapter coverage, native-object fixtures, and LLVM tool-path test metadata. Build activation and NativeAOT R8 policy remain in later layers.

Co-authored-by: Copilot App <[email protected]>
@simonrozsival
simonrozsival force-pushed the simonrozsival-typemap-nativeaot-object-adapter branch from c172dcb to 594de17 Compare September 24, 2026 14:59
Write extracted keys through an adjacent temporary file so failed writes preserve the previous output, and require clang before running native-object integration tests.

Co-authored-by: Copilot App <[email protected]>
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12825

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes — 0 errors, 1 warning.

The parser is defensive, preserves existing output on failure, and has broad malformed-input coverage. All 44 reported checks are green. The remaining gap is that the new LLVM tool boundary is tested only with synthesized metadata/relocation text; the added integration scaffold has no test methods yet, so real tool-output compatibility is not verified in this layer.

Generated by Android PR Reviewer for #12825 · copilot · gpt56 · 131.2 AIC · ⌖ 11.2 AIC · ⊞ 25.9K
Comment /review to run again

Generate real relocatable objects for every supported Android target triple and run the production task against llvm-readobj and llvm-objdump output.

Co-authored-by: Copilot App <[email protected]>
@simonrozsival

Copy link
Copy Markdown
Member Author

@dalexsoto review

@simonrozsival simonrozsival added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Sep 26, 2026

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NativeAOT object reader selects Java type-map groups from ELF relocation/EEType metadata rather than string heuristics, emits normalized deterministic keys, and preserves the previous output on failure. Real-object tests cover all four target architectures through the LLVM tool boundary; no blocker remains in this intentionally unactivated layer.

@simonrozsival
simonrozsival merged commit fb41d48 into main Sep 26, 2026
44 checks passed
@simonrozsival
simonrozsival deleted the simonrozsival-typemap-nativeaot-object-adapter branch September 26, 2026 14:57
simonrozsival added a commit that referenced this pull request Oct 4, 2026
## Why

Layer 3 of 4 in native stack #12893, depending on #12825. The retained typemap adapters need a shared incremental MSBuild pipeline that consumes exact producer outputs rather than discovering stale or pre-trim candidates.

## Changes

- Add `Microsoft.Android.Sdk.TypeMap.Proguard.targets` with shared key extraction, class/member rule generation, NativeAOT opt-in, complete-configuration override handling, and incremental output tracking.
- Return exact per-RID native-object and LLVM tool paths plus the complete linked typemap inventory from the assembly-resolution inner build.
- Expand the standalone `TypeMapProguardTests` to exercise actual production tasks and explicit imports of the new pipeline.

The pipeline is available but **not imported by product targets until Layer 4**. This layer does not change existing R8/D8 configuration, resource flags, legacy typemap gates, or DGML behavior. All content is carved from the frozen feature snapshot; only the three assigned files change.

## Validation

36 passed, 0 failed, 0 skipped:

```sh
dotnet test tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests.csproj \
  --filter 'FullyQualifiedName~TypeMapProguardTests' \
  -p:_AndroidTreatWarningsAsErrors=true \
  -p:_NativeAotLlvmReadObjPath=<NDK>/toolchains/llvm/prebuilt/<host>/bin/llvm-readobj
```

Coverage includes real native object groups, RID unions, linked DLLs and empty stubs, runtime/mode/input changes, missing files, deleted outputs, relative task-assembly paths, complete overrides, and exact producer metadata. The run used the existing NDK `llvm-readobj` and adjacent `llvm-objdump`/`clang`, without a full SDK build or new tooling.

Three frozen test methods are temporarily deferred because they require the existing-target edits in Layer 4:

- `PlatformConfigurationSeparatesCoreClrOptimizationFromObfuscation`
- `TypemapInputsDoNotRequestGraphsOrChangeParallelism`
- `NdkDependencyRequiresNativeObjectOptIn`

Restoring those methods in Layer 4 makes the test file identical to the frozen source.
simonrozsival added a commit that referenced this pull request Oct 7, 2026
## Summary

Layer 4 of 4 in native stack #12893, depending on #12827 (`simonrozsival-typemap-msbuild-pipeline`).

Activate the shared retained-typemap rule pipeline so eligible CoreCLR and explicitly opted-in NativeAOT R8 builds derive Java class roots from retained typemap keys rather than keeping every ACW. This removes the dependency on large NativeAOT DGML graphs while preserving runtime/JNI roots and user-authored Java source retention.

- Import the shared MSBuild pipeline, wire R8 flags/resources and incremental inputs, and gate legacy configuration generation. CoreCLR keeps scoped JNI-facing members and optimization while disabling renaming; NativeAOT retains class-wide member preservation and remains opt-in for NDK-backed object inspection.
- Remove the old DGML producer, resource, diagnostics implementation, and graph-specific test fixtures; retain the interface-collection runtime test.
- Add R8, build, and scoped-member device coverage; restore the three deferred `TypeMapProguardTests` methods and their attributes (17 cases); document retention behavior and retired diagnostics.

## Validation

```sh
dotnet test tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests.csproj \
  -v minimal \
  --filter 'FullyQualifiedName~TypeMapProguardTests|FullyQualifiedName~ExtractTypeMapKeys' \
  -p:_AndroidTreatWarningsAsErrors=true \
  -p:_NativeAotLlvmReadObjPath="$NDK_LLVM_BIN/llvm-readobj"
```

Passed **294/294**, **0 skipped**: 53 pipeline/generator cases (including all 17 restored cases) and 241 retained-key adapter cases. Used the supplied existing Android NDK `llvm-readobj` and adjacent `llvm-objdump`/`clang` executables read/execute-only.

All six changed XML targets/resources parse successfully with `xmllint --nonet --noout`, including XML comments. `git diff --check` passes.

**Not run:** `Xamarin.Android.Build.Tests` R8/build cases and `MSBuildDeviceIntegration` device cases. This worktree has neither a local built Android SDK (`bin/Debug/dotnet/dotnet` or Release) nor the corresponding test assemblies. No full SDK bootstrap or new tooling installation was performed for this frozen split.

## Frozen split integrity

Exactly 28 paths, +601/-1081, relative to #12827. The final tree is exactly `168821457836836b55368bb7ad53c2bfd9833cb4`: frozen source `87bd3f8a8b512859339e5446c1f86709a6c28c3b` with the base XA1037 documentation/localization/JavaSourceUtils changes preserved. `TypeMapProguardTests.cs` is byte-identical to the frozen source. No non-English localization files changed.

Native stack #12893 is registered with #12824, #12825, #12827, and this PR in order. The closed LLVM adapter #12823 is not in this stack.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

r8-rules Typemap-derived ProGuard and R8 rules ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants