Repository navigation
[tests] Prune unsupported Android cases and consolidate redundant tests - #12930
Conversation
Generate only supported CoreCLR and NativeAOT build configurations, omit test cases for runtime behaviors that cannot apply, and retain tests disabled for open issues. Co-authored-by: Copilot App <[email protected]>
Retain every invalid-input and UTF-8 member-ID assertion while merging related cases, remove vacuous or compiler-generated checks, and make JavaList.RemoveAt exercise the named API. Co-authored-by: Copilot App <[email protected]>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
One or more issues must be addressed before approval.
Review effort: Lite
Findings: 3
Open (3)
What changed in this PR
This PR prunes unsupported Android runtime combinations and consolidates redundant tests while preserving meaningful assertions.
Changes:
- Adds a shared supported runtime configuration source.
- Removes unsupported NativeAOT and redundant test cases.
- Repairs and consolidates selected test coverage.
| File | Description |
|---|---|
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/RunnerIntegrationTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/JdkVersionInfoTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/JdkInstallerTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/EmulatorRunnerTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/DownloadUtilsTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/AvdManagerRunnerTests.cs | Updated as part of this pull request. |
| tests/Xamarin.Android.Tools.AndroidSdk-Tests/AdbRunnerTests.cs | Updated as part of this pull request. |
| tests/MSBuildDeviceIntegration/Tests/MonoAndroidExportTest.cs | Updated as part of this pull request. |
| tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs | Updated as part of this pull request. |
| tests/MSBuildDeviceIntegration/Tests/DebuggingTest.cs | Updated as part of this pull request. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Interop/JavaListTest.cs | Updated as part of this pull request. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/MemoryStreamPoolTests.cs | Updated as part of this pull request. |
| tests/Microsoft.Android.Build.BaseTasks-Tests/HexUtilitiesTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/XASdkTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/WearTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/BaseTest.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/TrimmableTypeMapBuildTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/ManifestTest.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildWithLibraryTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest3.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssetPackTests.cs | Updated as part of this pull request. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs | Updated as part of this pull request. |
| src/Microsoft.Android.Sdk.Analysis/Tests/DNAS0001Tests.cs | Updated as part of this pull request. |
| src/Microsoft.Android.Sdk.Analysis/Tests/DNAA0001Tests.cs | Updated as part of this pull request. |
| external/Java.Interop/tests/Java.Interop-Tests/Java.Interop/SelfRegistrationTests.cs | Updated as part of this pull request. |
| external/Java.Interop/tests/Java.Interop-Tests/Java.Interop/JniTypeUtf8Test.cs | Updated as part of this pull request. |
Parameterize invalid-input tests so each null, empty, and whitespace value runs independently; restore separate JNI member-kind checks so one failure cannot mask the others. Co-authored-by: Copilot App <[email protected]>
|
/review |
|
❌ Android PR Reviewer failed. Please review the logs for details. Warning Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding. What happenedThe threat detection engine failed to produce results. Review the workflow run logs for details.
|
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ LGTM — 0 errors, 0 warnings, 1 suggestion.
The runtime-matrix pruning matches the supported configurations, the follow-up parameterization preserves independent invalid-input and JNI coverage, and the removed tests are redundant or vacuous as described. CI is green across all 44 reported checks. I left one non-blocking suggestion to preserve complete assertion reporting in the consolidated JDK discovery test.
Generated by Android PR Reviewer for #12930 · copilot · gpt56 · 755 AIC · ⌖ 11.1 AIC · ⊞ 25.9K
Comment /review to run again
| return; | ||
| } | ||
|
|
||
| Assert.That (versions.Select (v => v.MajorVersion), Does.Contain (21), "Should contain JDK 21"); |
There was a problem hiding this comment.
🤖 💡 Testing — Please consider wrapping this check and the per-version shape checks below in Assert.Multiple. Consolidating the former independent JDK 21 test avoids a second network request, but a missing JDK 21 now aborts this method before malformed display names or download URLs are reported; multiple assertions preserve the consolidation while keeping all failures visible.
Rule: Preserve independent test coverage
Purpose
Prune test cases that cannot run in a supported configuration and consolidate low-value tests without discarding meaningful coverage. Tests disabled for still-open bugs are retained rather than silently retired. Separate commits cover runtime-matrix pruning, redundant-test cleanup, and follow-up changes to preserve independent failure reporting.
Generate only supported runtime/release combinations
BaseTest.ValidRuntimeConfigurationssupplies the three valid combinations: CoreCLR Debug, CoreCLR Release, and NativeAOT Release. These 39 active methods use a single NUnitValueSourcefor the correlated(isRelease, runtime)pair; NUnit still combines that pair with each method's other[Values]dimensions. This prevents generating NativeAOT Debug cases that immediately calledIgnoreUnsupportedConfiguration. NUnit discovery found 147 runnable tuple-derived cases across the host and device assemblies, with no NativeAOT Debug case in these methods.AndroidUpdateResourcesTest.csDesignTimeBuild,CheckOldResourceDesignerIsNotUsed,CheckOldResourceDesignerWithWrongCasingIsRemoved,CheckThatXA1034IsRaisedForInvalidConfigurationAssetPackTests.csBuildLibraryWithAssetPack,BuildApplicationWithAssetPackThatHasInvalidName,BuildApplicationWithAssetPackOutsideProjectDirectory,BuildApplicationWithAssetPackOverrides,BuildApplicationWithAssetPackBuildTest.csCheckItemMetadata,CheckLogicalNamePathSeperators,Desugar,BuildApplicationWithJavaSourceUsingAndroidXBuildTest2.csBuildBasicApplication,BuildBasicApplicationThenMoveIt,XA0141ErrorIsRaised,CheckTimestamps,BuildApplicationAndClean,BuildApplicationWithLibraryAndClean,CustomApplicationClassAndMultiDexBuildWithLibraryTests.csBuildWithNativeLibraries,BuildLibraryWhichUsesResourcesIncrementalBuildTest.csGenerateJavaStubsAndAssemblyManifestTest.csModifyManifestPackagingTest.csCheckAppBundleTasks/LinkerTests.csWarnAboutAppDomainsTrimmableTypeMapBuildTests.csBuild_WithTrimmableTypeMap_Succeeds,Build_WithTrimmableTypeMap_IncrementalBuildWearTests.csBasicProjectXASdkTests.csDotNetPublishDefaultValuesDebuggingTest.csApplicationRunsWithoutDebugger, retaining the native-library-extraction and embedded-dex dimensionsInstallAndRunTests.csDeployToDevice,ActivityAliasRuns,GlobalLayoutEvent_ShouldRegisterAndFire_OnActivityLaunch,JsonDeserializationCreatesJavaHandle,AppWithStyleableUsageRuns,TypeAndMemberRemapping,FixLegacyResourceDesignerStep,GradleFBProjThe already ignored, issue-linked
MicrosoftIntuneandDotNetInstallAndRunPreviousSdkretain their data and ignore markers; their open bugs were not silently retired.Stop generating NativeAOT cases where the tested behavior cannot apply
The CoreCLR cases remain. Each narrowed NativeAOT case already skipped because the artifact or mechanism being asserted does not exist there; unreachable skip/null-result paths were removed where applicable.
BuildTest.csApplicationJavaClassProperties,ApplicationIdPlaceholderBuildTest2.csBuildReleaseArm64;BuildBasicApplicationCheckPdb,BuildBasicApplicationCheckPdbRepeatBuild,BuildAppCheckDebugSymbolslinked/output, which ILC does not produce. The others check managed Debug PDB/symbol artifacts unsupported in NativeAOT Debug.BuildTest3.csNativeLibraryJniPreload_*methods:NoDuplicates,IncludeCustomLibraries,ExcludeSomeCustomLibraries,ExcludeAllCustomLibraries,AddSomeCustomLibrariesAndIgnoreAll,AddSomeCustomLibrariesAndIgnoreAllByName,IgnoreAll_PreservesRequired,DefaultsWorkBuildWithLibraryTests.csProjectDependencies,MultiTfmTransitiveReferenceIncrementalBuildTest.csAppProjectTargetsDoNotBreak,LinkAssembliesNoShrink,AfterILLinkAdditionalStepsIsSkippedOnSecondBuild;ChangePackageNamingPolicyLowercasenaming, intentionally unsupported with the trimmable typemap.PackagingTest.csMissingSatelliteAssemblyInLibraryTasks/LinkerTests.csAndroidUseNegotiateAuthentication,PreserveIX509TrustManagerSubclasses,PreserveServices,WarnWithReferenceToPreserveAttributeInstallAndRunTests.csNativeAssemblyCacheWithSatelliteAssemblies,FastDeployEnvironmentFiles,SkiaSharpCanvasBasedAppRunsMonoAndroidExportTest.csMonoAndroidExportReferencedAppStarts,ExportedMembersSurviveGarbageCollectionMono.Android.Exportis unsupported under NativeAOT. Both valid CoreCLR Debug/Release checks remain.Five existing sources were narrowed without removing their test methods:
BuildTest2.Get_BuildHasNoWarningsDataandGet_BuildHasTrimmerWarningsData: remove three NativeAOT Debug rows each, keeping seven and eleven runnable cases and their diagnostic assertions.InstallAndRunTests.Get_DotNetRun_Data: remove two always-skipped NativeAOT/LLVM-IR typemap cases, keeping five cases on supported typemaps.LinkerTests.Get_AndroidAddKeepAlivesData: stop generating NativeAOT cases for ILLink keep-alive checks, retaining CoreCLR and explicit trimmable-typemap compatibility cases.BuildTest.Get_EnvironmentVariablesData: remove unsupported NativeAOTLowercaseMD5, keeping the other environment-variable combinations for both runtimes.Tests ignored for still-open issues—including version-code fixtures awaiting supported ABIs, repetitive-build regressions, Intune, and broken runtime-specific paths—are not deleted.
Consolidate or repair low-signal tests without masking failures
Microsoft.Android.Build.BaseTasks-Tests/MemoryStreamPoolTests.csCreateStreamWriterandCreateBinaryWriter. Both wrote throughMemoryStreamPool.Sharedbut checked reuse in an unrelated local pool.FilesTests.CopyIfStreamChanged_MemoryStreamPool_StreamWriterand_BinaryWriteralready confirm return and reset on the same pool actually used.ReuseandPutDisposedremain.Microsoft.Android.Build.BaseTasks-Tests/HexUtilitiesTests.csGetHexValue_DefaultsToUpperCaseandWriteHex_Span_UpperCase; the exhaustive uppercase and every-byte tests already exercise their default arguments and values. Lowercase and boundary checks remain.Microsoft.Android.Sdk.Analysis/Tests/DNAA0001Tests.csDNAA0001DoesNotShow, which only feeds an empty source file to the analyzer; validApplicationsnippets still assert no diagnostic for real code.Microsoft.Android.Sdk.Analysis/Tests/DNAS0001Tests.csIDE0002IsNotSuppressed, which tests Roslyn's wrapper rather than our suppressor. The issue-linked ignoredIDE0002IsSuppressedremains so the suppressor coverage gap stays visible.Mono.Android-Tests/Java.Interop/JavaListTest.csCountassertion intoAdd, retaining its subsequent two-item check. RepairRemoveAt: previously it duplicatedInsertand never removed; it now callsRemoveAt(0)and checks the remaining item and count in both generic and non-generic fixtures.Xamarin.Android.Tools.AndroidSdk-Tests/AdbRunnerTests.cs[TestCase]cases for null, empty, and whitespace constructor paths; failure in one input can no longer mask the others. Remove threeAdbPortRuleequality, deconstruction, andToStringchecks that only exercise compiler-generated members of a declaration-onlyrecord; parsing tests still verify rule fields.Xamarin.Android.Tools.AndroidSdk-Tests/AvdManagerRunnerTests.csConstructor_AcceptsEnvironmentVariables, which checked only thatnewreturned non-null; the realListAvdsAsyncintegration test constructs a runner with environment settings and invokes it.Xamarin.Android.Tools.AndroidSdk-Tests/EmulatorRunnerTests.csXamarin.Android.Tools.AndroidSdk-Tests/DownloadUtilsTests.csParseChecksumFile_HashOnly_WithTrailingNewline;HashOnlyand the\nrow ofVariousLineEndingsalready cover the same path.Xamarin.Android.Tools.AndroidSdk-Tests/JdkInstallerTests.csDiscoverAsync_ReturnsVersions, retaining its returned-version shape checks while eliminating a second live network request.Xamarin.Android.Tools.AndroidSdk-Tests/JdkVersionInfoTests.csMutableProperties_CanBeSetandResolvedUrl_DefaultsToNull, which merely round-trip automatic property setters or check the CLR's default null value; constructor mapping and customToStringremain tested.Xamarin.Android.Tools.AndroidSdk-Tests/RunnerIntegrationTests.csSdkManagersetup/teardown.Java.Interop-Tests/Java.Interop/JniTypeUtf8Test.cs[Test]methods for instance/static method IDs and instance/static field IDs. An initial merge ran them sequentially and hid later coverage when an earlier assertion failed; the review follow-up restored independent execution and failure reporting. All four original UTF-8-vs-string comparisons remain.Java.Interop-Tests/Java.Interop/SelfRegistrationTests.csLocal validation and limitation
make prepare && make allcompleted. An initial fresh-worktree build lacked API 37.1/37.2 reference assemblies; after locally building those API-specificMono.Androidoutputs,make allsucceeded without changing build-system files.BuildBasicApplication9,BuildHasNoWarnings7,BuildHasTrimmerWarnings11, deviceApplicationRunsWithoutDebugger12, andDotNetRun5; ignored Intune cases remain discoverable.MonoAndroidExportReferencedAppStarts4 passed.DotNetRuncases fail locally because a second physical device was attached alongside the emulator. The .NET SDKdotnet rundevice chooser requires an explicit--deviceargument in that situation;ANDROID_SERIALandADB_TARGETpin direct adb commands but do not override that chooser. These tests are not represented as passing. The five supported cases remain discoverable for the single-emulator CI lane.