Add-Type failed to create a type that implements IWebProxy

This is a regression introduced by .NET Core 1.1 preview release libraries. It also happens on alpha.12 release.

Steps to reproduce

$RequiredAssembliesForInternalWebProxy = @( [System.Net.IWebProxy].Assembly.FullName, 
                                            [System.Uri].Assembly.FullName )

$SourceForInternalWebProxy = @" 
using System; 
using System.Net;

namespace Microsoft.PowerShell.Commands.PowerShellGet 
{ 
    /// <summary>
    /// Used by Ping-Endpoint function to supply webproxy to HttpClient
    /// We cannot use System.Net.WebProxy because this is not available on CoreClr
    /// </summary>
    public class InternalWebProxy : IWebProxy
    {
        Uri _proxyUri;
        ICredentials _credentials;

        public InternalWebProxy(Uri uri, ICredentials credentials)
        {
            Credentials = credentials;
            _proxyUri = uri;
        }

        /// <summary>
        /// Credentials used by WebProxy
        /// </summary>
        public ICredentials Credentials
        {
            get
            {
                return _credentials;
            }
            set
            {
                _credentials = value;
            }
        }

        public Uri GetProxy(Uri destination)
        {
            return _proxyUri;
        }

        public bool IsBypassed(Uri host)
        {
            return false;
        }
    }
} 
"@

Add-Type -ReferencedAssemblies $RequiredAssembliesForInternalWebProxy `
         -TypeDefinition $SourceForInternalWebProxy `
         -Language CSharp `
         -ErrorAction SilentlyContinue

Expected behavior

Succeeds

Actual behavior

Add-Type : Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more
information.
At line:1 char:1
+ Add-Type -ReferencedAssemblies $RequiredAssembliesForInternalWebProxy ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Add-Type], ReflectionTypeLoadException
    + FullyQualifiedErrorId : System.Reflection.ReflectionTypeLoadException,Microsoft.PowerShell.Commands.AddTypeCommand

PS D:\>
PS D:\> $Error[0].Exception.LoaderExceptions
Method 'GetProxy' in type 'Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy' from assembly 'nxg304ea.wnz,
Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSVersion                      6.0.0-alpha
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
GitCommitId                    v6.0.0-alpha.12-104-g67e9bf6376b90c289d725c904280a2439de4dd3b
PSEdition                      Core
BuildVersion                   3.0.0.0
CLRVersion
SerializationVersion           1.1.0.1

Manikyam Bavandla (@bmanikm)

Manikyam Bavandla (@bmanikm) and I talked about why we didn't catch it earlier, and we find

PowerShellGet tests in PowerShellGet repo are not running on the latest powershell core release -- it's still running on alpha.11 in PowerShellGet repo.
This issue affects scenarios related to proxy, and only throws exception when explicitly importing PowerShellGet -- when it's loaded by auto-loading, the exception gets hidden. It's obvious that we don't have test explicitly importing this module, and the cmdlets we run in tests don't use proxy.

As a workaround, we can try to leverage PS classes, instead of Add-Type

Dongbo Wang (@daxian-dbw)

Here is what I found, when decompiled the assembly generated by Add-Type vs dotnet cli.

GetProxy() dotnet (good)

.method private final hidebysig newslot virtual 
	instance class [System.Runtime]System.Uri System.Net.IWebProxy.GetProxy (
		class [System.Runtime]System.Uri destination
	) cil managed 
{
	.override method instance class [System.Runtime]System.Uri [System.Net.Primitives]System.Net.IWebProxy::GetProxy(class [System.Runtime]System.Uri)
	// Method begins at RVA 0x2090
	// Code size 12 (0xc)
	.maxstack 1
	.locals init (
		[0] class [System.Runtime]System.Uri
	)

	IL_0000: nop
	IL_0001: ldarg.0
	IL_0002: ldfld class [System.Runtime]System.Uri Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy::_proxyUri
	IL_0007: stloc.0
	IL_0008: br.s IL_000a

	IL_000a: ldloc.0
	IL_000b: ret
} // end of method InternalWebProxy::System.Net.IWebProxy.GetProxy

GetProxy() Add-Type (bad)

.method private final hidebysig newslot virtual 
	instance class [System.Private.Uri]System.Uri System.Net.IWebProxy.GetProxy (
		class [System.Private.Uri]System.Uri destination
	) cil managed 
{
	.override method instance class [System.Private.Uri]System.Uri [System.Net.Primitives]System.Net.IWebProxy::GetProxy(class [System.Private.Uri]System.Uri)
	// Method begins at RVA 0x2090
	// Code size 12 (0xc)
	.maxstack 1
	.locals init (
		[0] class [System.Private.Uri]System.Uri
	)

	IL_0000: nop
	IL_0001: ldarg.0
	IL_0002: ldfld class [System.Private.Uri]System.Uri Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy::_proxyUri
	IL_0007: stloc.0
	IL_0008: br.s IL_000a

	IL_000a: ldloc.0
	IL_000b: ret
} // end of method InternalWebProxy::System.Net.IWebProxy.GetProxy

The only difference is that System.Uri lives in System.Private.Uri assembly (implementation assembly) and not in the reference assembly (System.Runtime)

Since we currently don't ship reference assemblies together with PowerShell, we cannot make Add-Type compile against them.

That could potentially affect other inheritance cases.
Still, it's not clear how to mitigate it and why it worked before and suddenly stopped now.

Dongbo Wang (@daxian-dbw) can you follow-up about it with dotnet team, please?

Note: the generated IL is the same in alpha.11, where Add-Type version works.

Note: here is the IWebProxy disasm (also uses System.Runtime)

.class interface public auto ansi abstract System.Net.IWebProxy
{
	// Methods
	.method public hidebysig newslot abstract virtual 
		instance class [System.Runtime]System.Uri GetProxy (
			class [System.Runtime]System.Uri destination
		) cil managed 
	{
	} // end of method IWebProxy::GetProxy

	.method public hidebysig newslot abstract virtual 
		instance bool IsBypassed (
			class [System.Runtime]System.Uri host
		) cil managed 
	{
	} // end of method IWebProxy::IsBypassed

	.method public hidebysig specialname newslot abstract virtual 
		instance class System.Net.ICredentials get_Credentials () cil managed 
	{
	} // end of method IWebProxy::get_Credentials

	.method public hidebysig specialname newslot abstract virtual 
		instance void set_Credentials (
			class System.Net.ICredentials 'value'
		) cil managed 
	{
	} // end of method IWebProxy::set_Credentials

	// Properties
	.property instance class System.Net.ICredentials Credentials()
	{
		.get instance class System.Net.ICredentials System.Net.IWebProxy::get_Credentials()
		.set instance void System.Net.IWebProxy::set_Credentials(class System.Net.ICredentials)
	}

} // end of class System.Net.IWebProxy

I verified that it's a dotnet core (not corefx) regression: if I copy-replace System.Private.CoreLib.dll in alpha.11 to the newer version, I get exactly the same errors.

I just tried it on the latest build from master and it's gone. O_o

Name                           Value                                                                                                   
----                           -----                                                                                                  
PSVersion                      6.0.0-alpha                                                                                            
PSEdition                      Core                                                                                                   
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                
BuildVersion                   3.0.0.0                                                                                                
GitCommitId                    v6.0.0-alpha.13-25-g8abb6c3a94d304af0dd7611c865b1b2ec6460113                                           
CLRVersion                                                                                                                            
WSManStackVersion              3.0                                                                                                    
PSRemotingProtocolVersion      2.3                                                                                                    
SerializationVersion           1.1.0.1

I was trying to repro the problem outside the PowerShell context, but I could not. :(
Both dotnet run and dotnet publish are able to load the types from the assembly.

Maybe somebody else may be able to do it.

project.json

{
    "name": "webproxy",
    "version": "1.0.0-*",

    "buildOptions": {
        "emitEntryPoint": true
    },

    "frameworks": {
        "netcoreapp1.0": {
            "dependencies": {
                "System.Net.Primitives": "4.3.0-preview1-24530-04",
                "Microsoft.NETCore.App": "1.1.0-preview1-001100-00",
                "Microsoft.CodeAnalysis.CSharp": "1.3.1-beta1-20160616-03",
                "System.Runtime.Loader": "4.3.0-preview1-24530-04"
            }
        }
    },

    "runtimes": {
        "osx.10.11-x64": { }
    }
}

Program.cs

using System; 
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using System.Reflection;
using System.IO;

class Program
{
    static string PROGRAM = @"
using System; 
using System.Net;

namespace Microsoft.PowerShell.Commands.PowerShellGet 
{ 
    /// <summary>
    /// Used by Ping-Endpoint function to supply webproxy to HttpClient
    /// We cannot use System.Net.WebProxy because this is not available on CoreClr
    /// </summary>
    public class InternalWebProxy : IWebProxy
    {
        Uri _proxyUri;
        ICredentials _credentials;

        public InternalWebProxy(Uri uri, ICredentials credentials)
        {
            Credentials = credentials;
            _proxyUri = uri;
        }

        /// <summary>
        /// Credentials used by WebProxy
        /// </summary>
        public ICredentials Credentials
        {
            get
            {
                return _credentials;
            }
            set
            {
                _credentials = value;
            }
        }

        Uri IWebProxy.GetProxy(Uri destination)
        {
            return _proxyUri;
        }

        public bool IsBypassed(Uri host)
        {
            return false;
        }
    }
}
";

    static void Main(string[] args)
    {
        string outputAssembly = args.Length > 0 ? args[0] : null;

        var syntaxTree = CSharpSyntaxTree.ParseText(PROGRAM);
        var references = new PortableExecutableReference[]
        {
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("System.Private.CoreLib")).Location),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("System.Net.Primitives")).Location),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("System.Private.Uri")).Location),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("System.Runtime")).Location),
        };

        foreach (var r in references)
        {
            Console.WriteLine("REFERENCE: " + r.FilePath);
        }

        CSharpCompilation compilation = CSharpCompilation.Create(
                System.IO.Path.GetRandomFileName(),
                syntaxTrees: new[] { syntaxTree },
                references: references,
                options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

        if (outputAssembly != null)
        {
            var emitResult = compilation.Emit(outputAssembly);
            Console.WriteLine("Assembly emitted in " + outputAssembly);
        }
        else 
        {
            using (var ms = new MemoryStream())
            {
                var emitResult = compilation.Emit(ms);

                foreach (var d in emitResult.Diagnostics)
                {
                    Console.WriteLine("ERROR: " + d.GetMessage());
                }

                if (emitResult.Success)
                {
                    ms.Flush();
                    ms.Seek(0, SeekOrigin.Begin);
                    
                    var assembly = System.Runtime.Loader.AssemblyLoadContext.Default.LoadFromStream(ms);
                    foreach (var type in assembly.ExportedTypes) {
                        Console.WriteLine("Type " + type.FullName);
                    }
                }
                else
                {
                    Console.WriteLine("emitResult no success");
                }
            }
        }
    }
}

Manikyam Bavandla (@bmanikm)

An update from Manikyam Bavandla (@bmanikm): there were 2 Add-Type uses in PowerShellGet previously. Now he merged those 2 into one C# code snippet and use only one Add-Type, and Add-Type started to work even though the same InternalWebProxy implementation is used. So PowerShellGet is unblocked for now (at least for alpha.14 release), but this bug still stands.

Manikyam Bavandla (@bmanikm), please add more details if I miss anything.

Adding another repro. Fairly sure this is a dotnet core issue, but it could be a dotnet core fix exposing an Add-Type issue? As above, we've found the issue doesn't repro in alpha11. I copied all the dotnet core DLLs from alpha11 into alpha15, and the issue doesn't repro. I don't know how type forwarding manifests in disassemblers, but ildasm, at least, shows [System.Private.Uri]System.Uri in ProgTest.dll (compiled in alpha11 and alpha15) instead of [System.Runtime]System.Uri that's present in other disassembled assemblies (e.g. System.Net.Http.dll's HttpClient).

Steps to reproduce

Contents of Test.cs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;

namespace ConsoleApp1
{
    public class Test
    {
        public void Go()
        {
            HttpRequestMessage _httpRequest = new HttpRequestMessage();
            _httpRequest.Method = new HttpMethod("POST");
            _httpRequest.RequestUri = new Uri("http://localhost:3000/cupcakes/chocolate");
        }
    }
}

In PowerShell alpha15:

$refs = @('System.Net.Http.dll','System.Private.Uri.dll','System.Runtime.dll')
$content = get-content "Test.cs" | out-string
write-verbose -message $content
Add-Type -TypeDefinition $content -ReferencedAssemblies $refs -OutputAssembly ProgTest.dll
Add-Type -Path .\ProgTest.dll -PassThru
$prog = New-Object ConsoleApp1.Test
$prog.Go()

Expected behavior

No error from test script above

Actual behavior

Exception calling "Go" with "0" argument(s): "Method not found: 'Void System.Net.Http.HttpRequestMessage.set_RequestUri(System.Uri)'."
At C:\Program Files\PowerShell\6.0.0.15\CompileProg.ps1:10 char:1
+ $prog.Go()
+ ~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : MissingMethodException

Move powershell to .NET Core 2.0 (#3556)

Fixed via #3556:

Ship netcoreapp reference assemblies with powershell to fix the issues in Add-Type (#2764). By default Add-Type will reference all those reference assemblies when compiling C# code. If -ReferenceAssembly is specified, then we search reference assemblies first, then the framework runtime assemblies, and lastly the loaded assemblies (possibly a third-party one that was already loaded).

Move powershell to .NET Core 2.0 (PowerShell#3556)

Navigation Menu

Repository navigation

Add-Type failed to create a type that implements IWebProxy #2764

Steps to reproduce

Expected behavior

Actual behavior

Environment data

GetProxy() dotnet (good)

GetProxy() Add-Type (bad)

1 remaining item

project.json

Program.cs

Steps to reproduce

Expected behavior

Actual behavior

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add-Type failed to create a type that implements IWebProxy #2764

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

daxian-dbw commented on Nov 22, 2016

vors commented on Dec 1, 2016

vors commented on Dec 3, 2016

GetProxy() dotnet (good)

GetProxy() Add-Type (bad)

vors commented on Dec 3, 2016

vors commented on Dec 3, 2016

1 remaining item

vors commented on Dec 3, 2016

project.json

Program.cs

daxian-dbw commented on Dec 6, 2016

brywang-msft commented on Feb 3, 2017

Steps to reproduce

Expected behavior

Actual behavior

daxian-dbw commented on Apr 17, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions