Repository navigation
[Android] Deserialization issue when consuming ASMX Web Service #80935
Description
Transferred customer issue from dotnet/android#7230
Description
I'm upgrading an existing Xamarin Traditional (Android) application into .NET 6 but I'm running into deserialization problems. Our Android app integrates with a .NET Framework 4.8 ASMX Web Service on the backend. This has been working fine for years but doesn't work after we move to .NET 6.
Reproduction Steps
I was able to create a repro project on my GitHub: https://github.com/adolfo1981/Net6AndroidWebServiceTest
My GitHub repro repo consists of the following projects:
- HelloAndroid (NET6 - .net6.0-android Target Framework Moniker)
- WebServiceServer (.NET Framework 4.8)
- WebServiceClient (.NET 6) --> referenced by HelloAndroid project to consume ASMX Web Services
- Create a Xamarin Android (NET 6) project similar to @jamesmontemagno https://github.com/jamesmontemagno/net6-mobile-samples/tree/main/HelloAndroid
- Create a .NET Framework 4.8 WebServiceServer application following instructions like this: https://www.c-sharpcorner.com/article/how-to-create-a-web-service-project-in-net-using-visual-studio/
- Create a .NET 6 (Class Library) WebServiceClient and add a Connected Service -> Web Service pointing to a wsdl or deployed asmx file
- From Main Activity create an instance of the generated soap client and call HelloWorldAsync method:
var soapClient = new WebServiceSoapClient(WebServiceSoapClient.EndpointConfiguration.WebServiceSoap);
//Async test
var helloResponse = await soapClient.HelloWorldAsync();
Expected behavior
Get string response from HelloWorldAsync() method
Actual behavior
Exception:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:HelloWorldResponse. The InnerException message was 'There was an error deserializing the object of type ServiceReference1.HelloWorldResponseBody. Unexpected end of file. Following elements are not closed: HelloWorldResult, HelloWorldResponse, Body, Envelope. Line 1, position 298.'. Please see InnerException for more details.
StackTrace:
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName)
at System.Runtime.Serialization.DataContractSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader, XmlObjectSerializer serializer) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 657
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 652
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameterPart(XmlDictionaryReader reader, PartInfo part, Boolean isRequest) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 521
Regression?
Yes, this worked in legacy Xamarin Android and should work in the .NET versions.
Known Workarounds
None
Configuration
.NET 6, .NET 7
Android (any architecture)
Other information
IMPORTANT:
The repro project points to my local IP (Bottom of Reference.cs file) and needs to be changed to IP of the user running the code. I could not use localhost because I was getting connectivity issues and only worked when setting my local IP.
Things that I've tried:
- Generating the Web Service Client with 'Reuse referenced assemblies' enabled and disabled and that had no effect. Kept getting deserialization issue
- Tried changing the HelloAndroid.csproj TargetFramework from .net6,0-android to .net6.0-android31.0 and nothing changed even after deleting /bin and /obj folders in this project
Activity
steveisok commentedon Jan 20, 2023
@simonrozsival Please triage this when you have a moment.
simonrozsival commentedon Jan 23, 2023
Triage: From the customer's report it's clear that there is an Android specific issue (the same code works in a .NET 6 console application) somewhere in the code that processes the response from the server. At a first glance this could be one of the following issues:
We should investigate further to identify the cause of the problem and fix it to align the behavior of all supported platforms.
The repro project didn't work on my PC out-of-the-box because one of the server project failed to load. I'll give it another try later this week.
adrianpbv commentedon Jan 27, 2023
Same problem here. I'm using a third party soap web service and when I try to make a request i get [DOTNET] Error in deserializing body of reply message for operation 'x'. I'm running the application on android. Also i notice that the tool dotnet-svcutil doesn't work properly using .net6-anyPlatform. So, when generating the file at the moment of adding the Web Service Reference maybe the System.Xml.Serialization package is not added to the project. The screenshot shows the Error List when I tried to add dotnet-svcutil to the project.
quiksilva7 commentedon Feb 6, 2023
Is there a solution to this yet? I am running into the exact same issue running .NET 7. Web Services run fine in a Windows environment, as soon as I try executing the same code in an Android environment I receive the error posted in this issue.
simonrozsival commentedon Feb 8, 2023
I was able to reproduce the bug in both .NET 6 and .NET 7. It appears to be a problem with response content decompression in AndroidMessageHandler. For some reason the decompressed XML is incomplete /cc @grendello @jonathanpeppers
There are two workarounds that might apply in some scenarios:
<UseNativeHttpHandler>false</UseNativeHttpHandler>grendello commentedon Feb 8, 2023
@simonrozsival if the decompression finished then the payload is probably "valid" (in the sense that the compressed data contains all that was sent/received), but it of course depends on the compression algorithm used. If it's GZip (most likely?), then the decompression may return a short result if one block of data fails to decompress. All the preceding blocks will still be decompressed. That begs the question - is the compressed data length (as reported by the
Content-Lengthheader) equal to how much data was actually fetched? You could test it by disabling automatic decompression in the client and do the decompression step separately, after receiving the content.If the length is different, then there might be a bug in that we somehow fail to read the last chunk of data from the service. If that's the case, then it might be a problem with the underlying Java client (a short read caused by something) or in our content stream handling. I find it hard to believe the latter is the case, since we've had this code for many years without any such issues, but nothing is impossible - perhaps some subtle semantics of stream reading changed between the mono/mono and dotnet BCLs?
Another test would be to see if changing the compression algorithm makes things work? The underlying Java client supports only GZip and Deflate IIRC, both should also be supported by the ASMX service server.
akoeplinger commentedon Feb 8, 2023
@adrianpbv the second issue you mentioned about
dotnet-svcutilis unrelated, looks like you added aPackageReferenceto it but that won't work since it is a tool that is supposed to be installed viadotnet tool install --global dotnet-svcutiland then used on the command line: https://learn.microsoft.com/en-us/dotnet/core/additional-tools/dotnet-svcutil-guide?tabs=dotnetsvcutil2xsimonrozsival commentedon Feb 8, 2023
@grendello Yes, it's
gzip. TheContent-Lengthis 311 and from what I can tell, WCF trims the decoded message to this length:It reminds me of another issue with the iOS native handler: dotnet/wcf#4903. I think this is a bug in WCF which doesn't correctly work with native handlers 🤔
grendello commentedon Feb 8, 2023
@simonrozsival hm, that sounds a bit like WCF takes the compressed data length and applies it to the decompressed content length?
simonrozsival commentedon Feb 8, 2023
@grendello It seems that way to me. If I remember correctly, SocketsHttpHandler doesn't expose the
Content-Lengthvalue through theHttpContentHeaders.ContentLengthproperty when the content is compressed. When theContentLengthproperty has a value (which is the case when either AndroidMessageHandler or NSUrlSessionHandler is used), it's used to limit the number of bytes that are read from the inputStream (GZipStream) in HttpResponseMessageHelper:I'm not very familiar with the WCF codebase so I might be wrong. I can try confirming my hypothesis next week. I just need to find a better way to debug the wcf code on Android.
/cc @mconnew
grendello commentedon Feb 8, 2023
@simonrozsival so it looks like WCF would work fine if we reset the content length to the size of data after decompression, however doing that would be both tricky (we wouldn't know the length until content were decompressed, and that may happen outside our control) and not entirely correct from the HTTP protocol POV. Looks like the correct fix is to teach WCF how to deal with compression.
mconnew commentedon Feb 9, 2023
@grendello, if WCF was handling the raw bytes from the connection and had to instantiate a GzipStream to decompress the contents, it would be examining the Content-Encoding header to do so. In that scenario, it would already have code that's aware about different compression encoding methods so it makes sense for it to handle the content-length header differently. That's not the case here though. HttpClientHandler (the implementation type we use) auto-magically handles compressed content for WCF. Whether it's GzipStream, DeflateStream, or something else, WCF is blissfully unaware that compression has been used.
Here's a list of Http implementations where WCF hasn't needed to special case compression:
Here's the list of Http implementations which require special handling:
Thinking beyond WCF, as the iOS and Android native wrappers still present as HttpClientHandler, I think it's a better developer experience that HttpClientHandler provides consistent behavior wherever it's being used as libraries which were never written for or tested on .NET MAUI are going to be used there. If a native implementation is specifically being used by instantiating a platform specific class, I think it's fair to say different implementations will behave differently. That's not the case here, the api being used is HttpClientHandler so whenever possible it should behave the same everywhere.
Every .NET HTTP implementation has behaved the exact same way on every platform for decades except the new .NET MAUI mobile platforms. I don't think it's WCF that needs to change behavior. I'm strongly of the opinion that it's .NET MAUI that has the bug. If you aren't going to provide the same behavior that HttpClientHandler has everywhere else, then don't provide an HttpClientHandler wrapper around the native implementation and only provide a differently named managed wrapper implementation and have HttpClientHandler use SocketsHttpHandler.
grendello commentedon Feb 10, 2023
@mconnew I both agree and disagree with what you wrote, but it doesn't really matter what I think because I found why the working client handlers work and the native wrappers don't. The working ones remove the
Content-Lengthheader from the response, as well as they remove the encoding referring to the compressed data. I think it's fair if we implement the same approach in the Android native handler wrapper as that'd be the fastest way to solve this issue. I'll work on a PR for this on the Xamarin.Android side.Tweak AndroidMessageHandler behavior for WCF support
StephenMolloy commentedon Feb 16, 2023
Looks like this is being worked on with dotnet/android#7785
[Mono.Android] Tweak AndroidMessageHandler behavior for WCF support (#…
psudodev commentedon Jul 19, 2023
Is there a workaround yet
I am facing similar issue on Maui Android on .NET 7
[0:] System.ServiceModel.ProtocolException: The number of bytes available is inconsistent with the HTTP Content-Length header. There may have been a network error or the client may be sending invalid requests.
at System.ServiceModel.Channels.HttpResponseMessageHelper.ReadBufferedMessageAsync(Task
1 inputStreamTask, TimeoutHelper timeoutHelper) in /_/src/System.ServiceModel.Http/src/System/ServiceModel/Channels/HttpResponseMessageHelper.cs:line 224 at System.ServiceModel.Channels.HttpResponseMessageHelper.ParseIncomingResponse(TimeoutHelper timeoutHelper) in /_/src/System.ServiceModel.Http/src/System/ServiceModel/Channels/HttpResponseMessageHelper.cs:line 58 at System.ServiceModel.Channels.HttpChannelFactory1.HttpClientRequestChannel.HttpClientChannelAsyncRequest.d__18[[System.ServiceModel.Channels.IRequestChannel, System.ServiceModel.Primitives, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext() in //src/System.ServiceModel.Http/src/System/ServiceModel/Channels/HttpChannelFactory.cs:line 1144at System.ServiceModel.Channels.RequestChannel.RequestAsync(Message message, TimeSpan timeout) in //src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/RequestChannel.cs:line 279
at System.ServiceModel.Channels.RequestChannel.RequestAsyncInternal(Message message, TimeSpan timeout) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/RequestChannel.cs:line 238
at System.Runtime.TaskHelpers.WaitForCompletionNoSpin[Message](Task
1 task) in /_/src/System.ServiceModel.Primitives/src/Internals/System/Runtime/TaskHelpers.cs:line 294 at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/RequestChannel.cs:line 227 at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Dispatcher/RequestChannelBinder.cs:line 107 at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/ServiceChannel.cs:line 760 at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/ServiceChannel.cs:line 717 at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(MethodCall methodCall, ProxyOperationRuntime operation) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/ServiceChannelProxy.cs:line 390 at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(MethodInfo targetMethod, Object[] args) in /_/src/System.ServiceModel.Primitives/src/System/ServiceModel/Channels/ServiceChannelProxy.cs:line 144 at generatedProxy_1.GetSysUTCDate(GetSysUTCDateRequest ) at SmartMobileClient.ISmartMobile.GetSysUTCDate(GetSysUTCDateRequest request) in C:\Users\uxp-abhijeet\source\SmartMobile\SmartMobileServiceProxy\Connected Services\SmartMobileServiceProxy\SmartMobile.cs:line 5317 at SmartMobileClient.GetSysUTCDate() in C:\Users\uxp-abhijeet\source\SmartMobile\SmartMobileServiceProxy\Connected Services\SmartMobileServiceProxy\SmartMobile.cs:line 5323 at SmartMobileApp.SmartMobileService.<GetSystemDateAsync>b__20_0() in C:\Users\uxp-abhijeet\source\SmartMobile\SmartMobileMaui\SmartMobileService.cs:line 266 at SmartMobileApp.SmartMobileService.<>c__DisplayClass10_01[[System.String, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].b__0() in C:\Users\uxp-abhijeet\source\SmartMobile\SmartMobileMaui\SmartMobileService.cs:line 105[0:] {}
[monodroid-assembly] open_from_bundles: failed to load assembly SmartMobileServiceProxy.XmlSerializers.dll
steveisok commentedon Jul 19, 2023
@psudodev It looks like dotnet/android#7785 was fixed, but I'm not sure it was backported to .NET 7. @grendello can you confirm?
steveisok commentedon Jul 19, 2023
Since this is not a runtime issue and has been fixed elsewhere, I'm going to close this.