Repository navigation
LibraryImportGenerator ModeThatUsesBidirectionalShape_Missing_AllMethods_ReportsDiagnostic failure in CI #73127
Description
Pipeline run: https://dev.azure.com/dnceng/public/_build/results?buildId=1913659&view=results
Test: https://dev.azure.com/dnceng/public/_build/results?buildId=1913659&view=ms.vss-test-web.build-test-results-tab&runId=49633240&resultId=190139&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab
Console log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-155c8e34ff754aeca1/LibraryImportGenerator.Unit.Tests/1/console.395cfa54.log?helixlogtype=result
Note that this is in the libraries-pgo pipeline, but no PGO is enabled in this particular scenario. There is both a failure in the net7.0-Linux-Release-arm64-CoreCLR_checked-nopgo job and the net7.0-windows-Release-x86-CoreCLR_checked-fullpgo_methodprofiling job.
LibraryImportGenerator.UnitTests.CustomMarshallerAttributeAnalyzerTests_StatefulValueShapeValidation.ModeThatUsesBidirectionalShape_Missing_AllMethods_ReportsDiagnostic [FAIL]
Microsoft.CodeAnalysis.Testing.Verifiers.EqualWithMessageException : Context: Diagnostics of test state
Expected diagnostic message to be "The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a one-parameter instance method named 'FromUnmanaged' that takes the 'unmanaged' value as a parameter and returns 'void'" was "The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a zero-parameter instance method named 'ToManaged' that returns 'ManagedType'"
Expected diagnostic:
// /0/Test0.cs(5,82,5,96): error SYSLIB1057: The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a one-parameter instance method named 'FromUnmanaged' that takes the 'unmanaged' value as a parameter and returns 'void'
VerifyCS.Diagnostic(CustomMarshallerAttributeAnalyzer.SYSLIB1057).WithSpan(5, 82, 5, 96).WithArguments("MarshallerType", "ManagedToUnmanagedRef", "ManagedType"),
Actual diagnostic:
// /0/Test0.cs(5,82): error SYSLIB1057: The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a zero-parameter instance method named 'ToManaged' that returns 'ManagedType'
VerifyCS.Diagnostic(CustomMarshallerAttributeAnalyzer.SYSLIB1057).WithSpan(5, 82, 5, 96).WithArguments("MarshallerType", "ManagedToUnmanagedRef", "ManagedType"),
Assert.Equal() Failure
Expected: The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a one-parameter instance method named 'FromUnmanaged' that takes the 'unmanaged' value as a parameter and returns 'void'
Actual: The type 'MarshallerType' specifies that it supports the 'ManagedToUnmanagedRef' marshal mode for 'ManagedType' but it does not provide a zero-parameter instance method named 'ToManaged' that returns 'ManagedType'
Stack Trace:
/_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Testing.Verifiers.XUnit/XUnitVerifier.cs(49,0): at Microsoft.CodeAnalysis.Testing.Verifiers.XUnitVerifier.Equal[T](T expected, T actual, String message)
/_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs(402,0): at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.VerifyDiagnosticResults(IEnumerable`1 actualResults, ImmutableArray`1 analyzers, DiagnosticResult[] expectedResults, IVerifier verifier)
/_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs(274,0): at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.VerifyDiagnosticsAsync(EvaluatedProjectState primaryProject, ImmutableArray`1 additionalProjects, DiagnosticResult[] expected, IVerifier verifier, CancellationToken cancellationToken)
/_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs(304,0): at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.RunImplAsync(CancellationToken cancellationToken)
/_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs(182,0): at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.RunAsync(CancellationToken cancellationToken)
/_/src/libraries/System.Runtime.InteropServices/tests/LibraryImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(44,0): at LibraryImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.VerifyAnalyzerAsync(String source, DiagnosticResult[] expected)
/_/src/libraries/System.Runtime.InteropServices/tests/LibraryImportGenerator.UnitTests/CustomMarshallerAttributeFixerTests_StatefulValueShapeValidation.cs(109,0): at LibraryImportGenerator.UnitTests.CustomMarshallerAttributeAnalyzerTests_StatefulValueShapeValidation.ModeThatUsesBidirectionalShape_Missing_AllMethods_ReportsDiagnostic()
--- End of stack trace from previous location ---
Activity
danmoseley commentedon Jul 31, 2022
#72888 seems to be the cause @jkoritzinsky
jkoritzinsky commentedon Aug 1, 2022
I've narrowed down the bug here. It's a non-determinism issue. The AnalyzerDriver runs concurrently, so it reports diagnostics in a non-deterministic order. The Roslyn testing SDK deterministically sorts the diagnostics, but they sort based on the location, id, and message arguments. The different diagnostics here only differ based on message template, description, and properties (not message arguments), so they end up comparing equal.
I'll put out a PR to the Roslyn SDK to provide an extensibility point here that we can use to fix this problem. In the meantime, we can disable the test against this issue.
jkoritzinsky commentedon Aug 1, 2022
It looks like there might be a mechanism to enforce a stronger match for the diagnostics so we don't need to enhance the SDK. I'll see if I can use it to fix this without updating the SDK.
Make diagnostic checks more reliable with more exact result matching.