Repository navigation
[MSBuild] Add retained typemap rule generation pipeline - #12827
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Generated typemap outputs are missing from Dalvik inputs, risking stale dex output, and manifest generation is not incremental.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
What changed in this PR
Adds a retained typemap ProGuard generation pipeline with producer metadata propagation and integration coverage.
Changes:
- Adds shared key extraction and ProGuard rule-generation targets.
- Propagates exact per-RID producer and tool paths.
- Expands pipeline and incremental-build tests.
| File | Summary |
|---|---|
tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests/Generator/TypeMapProguardTests.cs |
Tests production tasks and MSBuild integration. |
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Proguard.targets |
Defines the shared extraction and rule-generation pipeline. |
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.AssemblyResolution.targets |
Returns producer metadata from inner builds. |
c84b518 to
0742417
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
0742417 to
51fe93d
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
51fe93d to
ee4d1b9
Compare
ee4d1b9 to
4aa80bc
Compare
c0f720c to
c0f8419
Compare
|
@dalexsoto review |
dalexsoto
left a comment
There was a problem hiding this comment.
Two issues in the staged typemap pipeline need fixing before it can be imported:
-
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Proguard.targets:4,58-60,99-102: a trimmed R8 CoreCLR build withAndroidTypeMapImplementation=llvm-irselectsExtractTypeMapKeysFromLlvmIr, but no task by that name is compiled into$(_MicrosoftAndroidBuildTasksAssembly)at this head. This path fails with MSB4036 instead of generating typemap keys. Please supply and test the extractor, or keep this branch inactive until it exists. -
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Proguard.targets:37-45: the NativeAOT enabled and disabled targets shareproguard_project_references.cfg. On atrue->falsechange to_AndroidEnableTypemapR8Trimming, the ACW map and production_AndroidBuildPropertiesCachedo not change (the cache does not record this flag). The fallback therefore skips as up-to-date and retains the narrower enabled-mode rules, defeating the opt-out. The new transition test masks this by using a synthetic cache that does record the flag. Please include the mode in fallback invalidation and test against production-equivalent cache contents.
I treated the product import and Dalvik/R8 wiring as intentionally deferred to Layer 4; these are defects in the staged pipeline itself.
|
@dalexsoto Thanks for catching both of these. They are addressed now:
The fixes are in |
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
There was a problem hiding this comment.
⚠️ Needs Changes
Found 1 error and 1 warning in the new MSBuild pipeline: the complete ProguardConfigFiles override still executes an unused typemap generation path, and the member-rule target uses AfterTargets despite an available dependency extension point.
The producer metadata flow and focused incremental coverage are otherwise well designed. All 44 reported CI checks are green.
Generated by Android PR Reviewer for #12827 · copilot · gpt56 · 139.9 AIC · ⌖ 14 AIC · ⊞ 26.5K
Comment /review to run again
Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>
Skip generated typemap rules when custom ProGuard configuration replaces them, and order member generation through the Dalvik dependency chain. Co-authored-by: Copilot App <[email protected]>
Generate scoped member rules for NativeAOT and share mode-state inputs between modern and legacy ProGuard writers so mode round trips cannot reuse stale rules. Co-authored-by: Copilot App <[email protected]>
ef2c47c to
ea47cc1
Compare
Use the layout-aware Android SDK tools directory so the NativeAOT R8 regression locates build tasks in both local and CI workload installations. Co-authored-by: Copilot App <[email protected]>
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
There was a problem hiding this comment.
✅ LGTM
Findings: 0 errors · 0 warnings · 1 suggestion
The pipeline correctly keeps exact producer outputs, runtime/mode state, custom-configuration gating, and Dalvik incrementality aligned. The new target is intentionally not imported by product targets in this layer, matching the PR description. All 44 reported checks are green. I left one non-blocking inline suggestion to add equivalent end-to-end CoreCLR/R8 coverage for the NativeAOT constructor-retention regression test.
Generated by Android PR Reviewer for #12827 · copilot · gpt56 · 150 AIC · ⌖ 11.2 AIC · ⊞ 26K
Comment /review to run again
|
@dalexsoto review |
Migrate retained typemap target regressions into the existing NUnit build-task suite after main removed the standalone test project, preserving native-object and incremental-build coverage. Co-authored-by: Copilot App <[email protected]>
Include the latest legacy-tooling removal after the prepared main merge; retained typemap target and ProGuard generator regressions pass against the updated dependency layout. Co-authored-by: Copilot App <[email protected]>
dalexsoto
left a comment
There was a problem hiding this comment.
The previously reported extractor, NativeAOT member-retention, mode-invalidation, override and ordering concerns are resolved in current source. Product import and the stated Layer-4-dependent edits/tests remain accepted staging, not blockers. The complete eight-file review and separate integration pass identify one defect in the explicitly imported CoreCLR pipeline.
Keep linked-assembly extraction inactive when CoreCLR ILLink is disabled (TypeMap.Proguard.targets:125-128). With CoreCLR, PublishTrimmed=true, AndroidLinkTool=r8 and RunILLink=false, the new activation still invokes linked-DLL extraction, and the new inner-build inventory condition:85-91 still advertises typemap DLLs under $(IntermediateOutputPath)linked.
The repository's pinned SDK linker target excludes RunILLink=false, so on clean intermediates these linked DLLs are not produced. The existing CoreCLR publish adapter:317-320 can publish generated typemaps without linked copies; that does not create the linked paths advertised by this new inventory. Assembly extraction:33-44 opens those paths and reports missing files as XA4327, failing this non-linking configuration instead of leaving extraction inactive.
Exclude RunILLink=false from the CoreCLR activation and linked-inventory conditions and add a clean-intermediates case. Keep strict missing-input validation and exact linked producer contracts; do not substitute stale/pre-trim discovery. Do not apply this exclusion to NativeAOT, where ILC performs trimming.
This activation/producer mismatch is established by immutable source tracing, not an executed MSBuild reproduction. The observed CI failure in build1620088 is separately verified as a submodule checkout receive timeout before build/tests, and is not the reason for this source finding.
Gate linked typemap inventory and CoreCLR rule activation on RunILLink, preserve NativeAOT extraction, and cover both runtimes' JNI-only constructor retention through R8. Co-authored-by: Copilot App <[email protected]>
|
@dalexsoto Fixed the disabled-linker activation/producer mismatch in Both CoreCLR activation conditions ( I reproduced the defect with clean-output target fixtures before changing the production conditions: both CoreCLR activation cases and the disabled-linker inventory case failed. They now pass, asserting that no keys, manifests, class/member rules, or modern-policy activation are produced despite missing linked DLLs and a deliberately missing task assembly. Producer tests cover blank/true/false linker states, and a real NDK-backed NativeAOT case proves Validation: 46 focused target/generator cases passed with 0 skips. The separate CoreCLR/R8 coverage suggestion is addressed too: both CoreCLR and NativeAOT end-to-end builds pass and retain the JNI-only |
dalexsoto
left a comment
There was a problem hiding this comment.
The disabled-ILLink activation/producer mismatch is fixed, and the earlier NativeAOT retention, mode-state, override and ordering findings remain resolved. The declared Layer-3 product-import deferral is accepted. The complete current eight-file review and separate integration/completeness pass identify one remaining incremental-contract blocker.
Track CoreCLR opt-out as a Dalvik input change (TypeMap.Proguard.targets:125-129). After an enabled trimmed CoreCLR/R8 build, a no-clean rebuild with _AndroidEnableTypemapR8Trimming=false skips the modern configuration/class/member targets before their dependencies. The effective configuration/file list changes, but the only mode-file writer is NativeAOT-only (lines 26-38). CoreCLR therefore does not update a policy sentinel or register an inactive-mode change with Dalvik.
The production properties cache does not record this flag, and the resolved-assembly hash ignores changed producer metadata. The real Dalvik inputs contain user @(ProguardConfiguration), not the internal selected configuration list; removing the modern input registrations does not invalidate the existing timestamp-based Dalvik stamp. When the remaining producer/consumer inputs retain their timestamps, R8 can skip and leave classes.dex from the previously selected keep configuration. The baseline CoreCLR configuration writer can also remain up to date; this finding does not assume that opt-out necessarily generates fresh legacy contents. This is pinned static source proof, not an executed rebuild or a claim that every flag flip skips R8.
Persist the effective CoreCLR policy mode even when modern extraction is inactive, include that state file in Dalvik inputs, and add a no-clean enabled/disabled/enabled consumer regression using unchanged linked DLLs and production-like cache/input contracts. Do not invoke inactive extraction/tasks or require Layer-4 product imports to fix this staged-pipeline contract.
Review-completeness correction: this gap already existed at the previously reviewed 2b4d048e0fb03db209b38399b6271c6eb7599908 head and was missed in the earlier complete review. It was not introduced by da429; the separate disabled-ILLink fix is valid.
Persist the effective CoreCLR ProGuard policy independently of active extraction and include it in Dalvik inputs. Cover no-clean opt-out, disabled ILLink, and custom configuration round trips with unchanged linked assemblies and cache timestamps. Co-authored-by: Copilot App <[email protected]>
|
@dalexsoto Fixed the CoreCLR opt-out Dalvik invalidation gap in A new CoreCLR policy-state writer runs directly through I reproduced the reported defect before changing the target: the no-clean enabled-to-disabled invocation kept the previous Dalvik stamp. The regression now covers enabled/disabled/enabled with unchanged linked DLL and production-like cache timestamps, asserts the consumer's selected policy/member list on each transition, and checks that repeated same-mode invocations preserve both the policy-file timestamp and the Dalvik stamp. Disabled invocations use a deliberately missing modern task assembly and verify the key output remains unchanged. The same round-trip checks also cover Validation: 49 focused target/generator cases passed, 0 failed or skipped, including the existing NDK-backed NativeAOT cases. Target XML and |
dalexsoto
left a comment
There was a problem hiding this comment.
The CoreCLR opt-out invalidation blocker is resolved: the policy-state writer now runs directly in the outer Dalvik dependency chain even when modern generation is inactive, updates a surviving consumer input with write-only-when-different semantics, and registers it for Clean. Re-enabling policy also invalidates class generation; disabled-linker/full-override paths avoid modern task invocation, and NativeAOT's shared-state behavior is preserved.
The complete eight-file/current-hunk review, separate producer-to-actual-Dalvik integration pass and final additional-blocker sweep found no remaining high-confidence source blocker. The declared Layer-3/Layer-4 staging and previously accepted policy decisions remain accepted. This is immutable source/contract review,not independently executed private R8,NDK,build or device validation.
## Summary Layer 4 of 4 in native stack #12893, depending on #12827 (`simonrozsival-typemap-msbuild-pipeline`). Activate the shared retained-typemap rule pipeline so eligible CoreCLR and explicitly opted-in NativeAOT R8 builds derive Java class roots from retained typemap keys rather than keeping every ACW. This removes the dependency on large NativeAOT DGML graphs while preserving runtime/JNI roots and user-authored Java source retention. - Import the shared MSBuild pipeline, wire R8 flags/resources and incremental inputs, and gate legacy configuration generation. CoreCLR keeps scoped JNI-facing members and optimization while disabling renaming; NativeAOT retains class-wide member preservation and remains opt-in for NDK-backed object inspection. - Remove the old DGML producer, resource, diagnostics implementation, and graph-specific test fixtures; retain the interface-collection runtime test. - Add R8, build, and scoped-member device coverage; restore the three deferred `TypeMapProguardTests` methods and their attributes (17 cases); document retention behavior and retired diagnostics. ## Validation ```sh dotnet test tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests/Microsoft.Android.Sdk.TrimmableTypeMap.Tests.csproj \ -v minimal \ --filter 'FullyQualifiedName~TypeMapProguardTests|FullyQualifiedName~ExtractTypeMapKeys' \ -p:_AndroidTreatWarningsAsErrors=true \ -p:_NativeAotLlvmReadObjPath="$NDK_LLVM_BIN/llvm-readobj" ``` Passed **294/294**, **0 skipped**: 53 pipeline/generator cases (including all 17 restored cases) and 241 retained-key adapter cases. Used the supplied existing Android NDK `llvm-readobj` and adjacent `llvm-objdump`/`clang` executables read/execute-only. All six changed XML targets/resources parse successfully with `xmllint --nonet --noout`, including XML comments. `git diff --check` passes. **Not run:** `Xamarin.Android.Build.Tests` R8/build cases and `MSBuildDeviceIntegration` device cases. This worktree has neither a local built Android SDK (`bin/Debug/dotnet/dotnet` or Release) nor the corresponding test assemblies. No full SDK bootstrap or new tooling installation was performed for this frozen split. ## Frozen split integrity Exactly 28 paths, +601/-1081, relative to #12827. The final tree is exactly `168821457836836b55368bb7ad53c2bfd9833cb4`: frozen source `87bd3f8a8b512859339e5446c1f86709a6c28c3b` with the base XA1037 documentation/localization/JavaSourceUtils changes preserved. `TypeMapProguardTests.cs` is byte-identical to the frozen source. No non-English localization files changed. Native stack #12893 is registered with #12824, #12825, #12827, and this PR in order. The closed LLVM adapter #12823 is not in this stack.

Why
Layer 3 of 4 in native stack #12893, depending on #12825. The retained typemap adapters need a shared incremental MSBuild pipeline that consumes exact producer outputs rather than discovering stale or pre-trim candidates.
Changes
Microsoft.Android.Sdk.TypeMap.Proguard.targetswith shared key extraction, class/member rule generation, NativeAOT opt-in, complete-configuration override handling, and incremental output tracking.TypeMapProguardTeststo exercise actual production tasks and explicit imports of the new pipeline.The pipeline is available but not imported by product targets until Layer 4. This layer does not change existing R8/D8 configuration, resource flags, legacy typemap gates, or DGML behavior. All content is carved from the frozen feature snapshot; only the three assigned files change.
Validation
36 passed, 0 failed, 0 skipped:
Coverage includes real native object groups, RID unions, linked DLLs and empty stubs, runtime/mode/input changes, missing files, deleted outputs, relative task-assembly paths, complete overrides, and exact producer metadata. The run used the existing NDK
llvm-readobjand adjacentllvm-objdump/clang, without a full SDK build or new tooling.Three frozen test methods are temporarily deferred because they require the existing-target edits in Layer 4:
PlatformConfigurationSeparatesCoreClrOptimizationFromObfuscationTypemapInputsDoNotRequestGraphsOrChangeParallelismNdkDependencyRequiresNativeObjectOptInRestoring those methods in Layer 4 makes the test file identical to the frozen source.