Repository navigation
[HTTP3] [outerloop] Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection #95413
Description
Error Blob
{
"ErrorMessage": "Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection",
"BuildRetry": false,
"ErrorPattern": "",
"ExcludeConsoleLog": false
}Reproduction Steps
@dotnet/ilc-contrib @dotnet/ncl PTAL
Note: This issue might be related to #73290, but as I wasn't sure, I created this one so I could mark it as a KnownBuildError so we could track the number of hits.
- Affected PR: Add XUnitLogChecker to log libraries dumps, do not enable for NativeAOT tests #94868 via
/azp run runtime-nativeaot-outerloop - Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=483727
- Jobs:
- https://dev.azure.com/dnceng-public/public/_build/results?buildId=483727&view=logs&j=1a903bec-56c1-5f69-7b57-f0ab9c1e238e&t=2fed0c68-9f2b-551e-de27-060b59e2b30c&l=70
- https://dev.azure.com/dnceng-public/public/_build/results?buildId=483727&view=logs&j=abdbb26b-d534-5e94-28a6-1260271ac680&t=669f6d83-cb64-5444-422f-fe4972234da6
- Queues:
linux-arm64 Release NativeAOT_Libslinux-x64 Release NativeAOT_Libs
- Log files:
- https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-94868-merge-6a7c0f61e2f54aa0a5/System.Net.Http.Functional.Tests/3/console.4c62795f.log?helixlogtype=result
- https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-94868-merge-77f3e8adef9f43219d/System.Net.Http.Functional.Tests/3/console.0d77b66b.log?helixlogtype=result
- Output:
[FAIL] System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.ConnectionAttemptCanceled_AuthorityNotBlocked
System.Net.Http.HttpRequestException : Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection.
---- System.OperationCanceledException : The operation was canceled.
at System.Net.Http.HttpConnectionPool.ThrowGetVersionException(HttpRequestMessage, Int32, Exception) + 0x108
at System.Net.Http.HttpConnectionPool.<TrySendUsingHttp3Async>d__87.MoveNext() + 0x1fc
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext() + 0x614
--- End of stack trace from previous location ---
at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext() + 0x194
--- End of stack trace from previous location ---
at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext() + 0x400
--- End of stack trace from previous location ---
at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass29_0.<<ConnectionAttemptCanceled_AuthorityNotBlocked>b__1>d.MoveNext() + 0x46c
--- End of stack trace from previous location ---
at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task) + 0x2c
--- End of stack trace from previous location ---
at System.Threading.Tasks.TaskTimeoutExtensions.<WhenAllOrAnyFailed>d__10.MoveNext() + 0x2c0
--- End of stack trace from previous location ---
at System.Threading.Tasks.TaskTimeoutExtensions.<WhenAllOrAnyFailed>d__8.MoveNext() + 0xe8
--- End of stack trace from previous location ---
at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<ConnectionAttemptCanceled_AuthorityNotBlocked>d__29.MoveNext() + 0x1a8
--- End of stack trace from previous location ---
--- End of stack trace from previous location ---
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
at System.Net.Quic.ValueTaskSource.TryComplete(Exception) + 0x8c
at System.Net.Quic.ValueTaskSource.<>c.<TryInitialize>b__10_0(Object obj, CancellationToken cancellationToken) + 0x7c
at System.Threading.CancellationTokenSource.Invoke(Delegate, Object, CancellationTokenSource) + 0x94
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) + 0x15c
at System.Threading.CancellationTokenSource.Invoke(Delegate, Object, CancellationTokenSource) + 0x44
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) + 0x15c
at System.Threading.CancellationTokenSource.Invoke(Delegate, Object, CancellationTokenSource) + 0x44
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) + 0x15c
at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass29_1.<ConnectionAttemptCanceled_AuthorityNotBlocked>b__3(Object _, X509Certificate _, X509Chain _, SslPolicyErrors _) + 0x30
at System.Net.Quic.QuicConnection.SslConnectionOptions.ValidateCertificate(QUIC_BUFFER*, QUIC_BUFFER*, X509Certificate2&) + 0x1bc
at System.Net.Quic.QuicConnection.HandleEventPeerCertificateReceived(QUIC_CONNECTION_EVENT._Anonymous_e__Union._PEER_CERTIFICATE_RECEIVED_e__Struct&) + 0x2c
at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT&) + 0x94
at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE*, Void*, QUIC_CONNECTION_EVENT*) + 0x1a4
--- End of stack trace from previous location ---
at System.Net.Quic.ValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16) + 0x18
at System.Net.Quic.QuicConnection.<FinishConnectAsync>d__34.MoveNext() + 0x60c
--- End of stack trace from previous location ---
at System.Net.Quic.QuicConnection.<<ConnectAsync>g__StartConnectAsync|2_0>d.MoveNext() + 0x168
--- End of stack trace from previous location ---
at System.Net.Quic.QuicConnection.<<ConnectAsync>g__StartConnectAsync|2_0>d.MoveNext() + 0x480
--- End of stack trace from previous location ---
at System.Net.Http.ConnectHelper.<ConnectQuicAsync>d__3.MoveNext() + 0x1c4
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<GetHttp3ConnectionAsync>d__86.MoveNext() + 0x8ec
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<TrySendUsingHttp3Async>d__87.MoveNext() + 0x314
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext() + 0x614
--- End of stack trace from previous location ---
at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext() + 0x194
--- End of stack trace from previous location ---
at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext() + 0x140
Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=483727
Error message validated: Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 11/29/2023 5:31:56 PM UTC
Report
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 36 | 37 | 37 |
Activity
[-][NativeAOT] [HTTP3] Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection[/-][+][NativeAOT] [HTTP3] [outerloop] Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection[/+]jkotas commentedon Nov 29, 2023
@wfurt Do we need to disable this test when Quic is not available?
ManickaP commentedon Nov 29, 2023
This test has gotten as far as cert validation, so seems like MsQuic is present and it's not #73290.
More digging shows that this might have been broken by #94211 @rzikm. The failures seem relevant.
ManickaP commentedon Nov 29, 2023
I see, because now the OCE has wrong token (the linked one, not the user provided):
runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs
Line 79 in 35b866c
We then check it in blocked alt-svc:
runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs
Lines 932 to 936 in 35b866c
[-][NativeAOT] [HTTP3] [outerloop] Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection[/-][+][HTTP3] [outerloop] Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection[/+]wfurt commentedon Nov 29, 2023
There should not be reason why this impacts only AOT, right @ManickaP?
There is also #88832 @rzikm disabled recently.
ManickaP commentedon Nov 29, 2023
It doesn't impact only AOT, e.g.:https://dev.azure.com/dnceng-public/public/_build/results?buildId=480926&view=ms.vss-test-web.build-test-results-tab
The other issue you linked is unrelated, as Radek's change was in Quic.
rzikm commentedon Nov 30, 2023
Weird enough, I can't reproduce the failure locally. I will have to dig a bit deeper.