Repository navigation
System.Reflection.Emit.AssemblyBuilder.Save #15704
Description
AssemblyBuilder.Save and AssemblyBuilderAccess.RunAndSave isn't available in .NET Core, however coreclr seems to have the code to implement it but I looks conditionally compiled out.
Our use case in Castle DynamicProxy is to write out dynamically created assemblies to disk so we can run peverify over the assembly in unit tests. It also greatly helps writing out the assembly and opening it in ildasm to manually verify IL.
Is there any chance we can get this functionality that .NET Framework has introduced into .NET Core, or an alternative way of doing the same thing. This might not sound like a big deal if you aren't familiar with Castle DynamicProxy's internals, but it really is as it heavily reduces our confidence in both our code and the .NET runtime as well as reducing our ability to track down defects.
Activity
joshfree commentedon Nov 13, 2015
Related issue https://github.com/dotnet/coreclr/issues/1709
bartdesmet commentedon Jan 6, 2016
Another interesting omission is the lack of LambdaExpression.Compile accepting a MethodBuilder which tends to be very useful when combined with AssemblyBuilder.Save support.
jkotas commentedon Feb 28, 2016
Large chunks of Reflection.Emit code in the runtime are Windows-specific. It is not easy to just enable them for CoreCLR because of it would not work on Unix. In particular, the support for emitting debug information depends on unmanaged PDB writer that is very complex Windows-specific component that we have no plans to open source and bring x-plat.
We are also not happy about Reflection.Emit being integrated deeply into the runtime. Reflection.Emit should be a independent higher level component that generates PE file that the runtime loads as any other assembly (note that runtime supports loading assemblies from byte array, so the PE file does not have to be persisted on disk). IKVM.Reflection.Emit done by Jeroen Frijters proved that it is pretty doable and can actually work much better than the current Reflection.Emit built into the runtime.
The plan to address the Reflection.Emit is:
jonorossi commentedon Feb 29, 2016
@jkotas many thanks for the explanation here, it makes complete sense. I'm glad you guys are planning to provide a more general purpose object model around the heart of .NET, IL. I'd seen the portable PDB reader/writer and some metadata object model appear in Roslyn, they look great. In the past I've used Cecil for the exact reason the .NET Framework API is so heavily tied into the runtime, glad to see more stuff being pulled out of the runtime.
tmat commentedon Feb 29, 2016
FYI @ManishJayaswal @jaredpar
karelz commentedon Feb 28, 2017
Next step: We need API proposal with deep analysis how implementable it is. The old Desktop API might be an option, likely not a good one.
This is quite hard to do and requires lots of experience.
KevinRansom commentedon May 30, 2017
@karelz @jkotas --- What is the status of this feature? When can we expect to see dynamic codegen with persistence on the coreclr?
karelz commentedon May 30, 2017
There is currently no plan (milestone Future + my comment above) - area owners @atsushikan @DnlHarvey @joshfree can provide more details.
stakx commentedon Jan 6, 2018
@atsushikan @DnlHarvey @joshfree, one question about the plan that @jkotas laid out above:
I do not fully understand the plan, in particular with regard to dynamic assemblies (
AssemblyBuilder.DefineDynamicAssembly) and their ability to be incrementally updated / augmented with new dynamic types. How would this still be possible if assemblies must first be fully written out as a byte stream or file, then be loaded as a whole?I imagine one would have to write each dynamically generated type to a single assembly... but wouldn't this in turn interfere with
internaltype visibility and type member accessibility between generated types that should all be in the same assembly, and with stuff like[assembly: InternalsVisibleTo("DynamicallyGeneratedAssembly")(on which e. g. DynamicProxy currently depends to be able to proxy internal user types)?Would it be possible to load more than one assembly having the exact same name (whether weak or strong), so that each such single-type assembly can have a
[assembly: InternalsVisibleTo]attribute for its own name, such that types in the separate assemblies can reference each other'sinternals?jkotas commentedon Jan 6, 2018
Yes, there would need to be a runtime hook for this.
Conceptually, each type addition in Reflection.Emit produces a new assembly that contains all existing metadata (with all existing tokens preserved) plus the new type. Then the runtime is asked to update the metadata of the already loaded assembly to the new version.
Sending the whole assembly over for each edit is not efficient for large assemblies with small edits. It would be better to just send the metadata delta with the new stuff over. Fortunately, this exists in the runtime for edit and continue already. Look for https://github.com/dotnet/coreclr/search?q=ApplyEditAndContinue.
So exposing the ApplyDelta metadata API for the standalone Reflection.Emit implementation should do the trick here.
masonwheeler commentedon May 7, 2018
The existence of cross-platform Roslyn suggests that this isn't a particularly intractable problem, at a high level at least.
Could this please be made a high priority? The inability to use Reflection.Emit is currently my dotnet/corefx#1 blocker preventing me from migrating existing work to Core, and as such it's far more urgent, from my perspective at least, than the development of new features.
zgramana commentedon May 7, 2018
Here's a good starting point for a non-Windows implementation (and it's rather mature): https://github.com/mono/mono/tree/master/mcs/class/corlib/System.Reflection.Emit
119 remaining items
acaly commentedon Jan 22, 2024
Thanks for the reply.
I know there is the new package, but it's different from the .NET Framework
GenerateSerializermethod, which allows me to generate a serializer assembly of a specified assembly, with specific types and mappings.In one of the projects that I am working on, we have an obfuscator that extracts types that need to be serialized and generate the corresponding serializer assembly, which gets merged into the main assembly and obfuscated together. The current package based method is very hard to be used in this process. Our workaround is to use a .NET Framework app to generate the serializer assembly, which is rewritten to target .NET.
This workaround currently works fine, but obviously it's risky. I think a source-generator based method or a public generator method should be more helpful. I'm not sure whether there are undocumented methods or better solutions for this though.
This is probably not a good place to continue this topic. If you think it's necessary I can open a new issue.
buyaa-n commentedon Feb 15, 2024
Thank you @kg for sending assemblies, I have regenerated the assembly with .NET Core implementation and found many issues that now fixed and will be ported with .NET 9 preview 2.
buyaa-n commentedon Feb 15, 2024
The feature is now available in https://dotnet.microsoft.com/en-us/download/dotnet/9.0, please give it a try and give us a feedback. We also adding bunch of fixes in preview 2 mostly related to complex generic types/methods
Here is related doc: https://learn.microsoft.com/dotnet/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder#persistable-dynamic-assemblies-in-net-core
mjsabby commentedon Feb 15, 2024
Will Portable PDB support be there as well in this preview?
buyaa-n commentedon Feb 15, 2024
Entry point and PDB not included in preview 1. Entry point will be added by preview 3. PDB support timeline not clear yet.
jgh07 commentedon Feb 17, 2024
There are a few methods still missing unrelated to PDB generation:
There might be alternatives that I'm unaware of. If not, will these methods also be supported in the future?
masonwheeler commentedon Feb 17, 2024
I really hope it makes it in by release. At its core, this is a tool for compiler developers, and a compiler that can't produce debug builds is severely disadvantaged.
Martin1994 commentedon Feb 18, 2024
Portable PDB is not blocked.
System.Reflection.Metadata.Ecma335is for this use case (not documented too well though IMHO). See https://github.com/Martin1994/ikvm/blob/18b1fce8265a3f128f4742b506f8e10e0cae0f51/src/IKVM.Reflection/Impl/PortablePdbWriter.csSystem.Reflection.Emitis more for dynamic emitting use cases, such as IronPython I believe.masonwheeler commentedon Feb 19, 2024
@Martin1994 That link looks kind of useful, but it looks like it only works if you already have an existing
System.Reflection.Metadataemitter.IIRC IronPython style dynamic emitting was covered by the DLR, which was released with .NET Framework 4.0.
System.Reflection.Emit, and the ability to for compiler writers to use it to save new assemblies, has been around since 1.0.buyaa-n commentedon Mar 18, 2024
FYI: We took completely different approach for adding
EntryPointsupport and setting other options, we are providing all metadata information produced with Reflection.Emit APIs (MetadataBuilder and ILStreams) so that user could embed them into the corresponding section ofPEBuidler. This will let the customers handle their assembly building process by themselves using the PEHeaderBuilder and ManagedPEBuilder options. Because the MetadataBuilder and BlobBuilder types are not accessible from within CoreLib we decided to make thePersistedAssemblyBuildertype public and reintroduce the related new APIs there, remove the new APIs from the baseAssemblyBuildertype.namespace System.Reflection.Emit; public abstract partial class AssemblyBuilder { // Previously approved new APIs - public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null); - public void Save(Stream stream); - public void Save(string assemblyFileName); - protected abstract void SaveCore(Stream stream); } +public sealed class PersistedAssemblyBuilder : AssemblyBuilder { + public PersistedAssemblyBuilder(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null); + public void Save(Stream stream); + public void Save(string assemblyFileName); + public MetadataBuilder GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder mappedFieldData); }See more details and usage example from the API proposal. This change will be available in preview 3.
buyaa-n commentedon Mar 18, 2024
Further the PDB support will be handled similar way as one would set
EntryPointinPEBuidler, here is the initial API proposal and usage example, please give your feedbacks.With that I am closing this issue as alternate for the
AssemblyBuilder.Saveis implementation is done