Proposal for high-performance codegen-less Reflection factory APIs

@jkotas

Background

There are certain scenarios today - largely involving activation, serialization, and DI - where library authors perform codegen in order to perform operations on arbitrary types. The primary reason for this is performance. The standard Reflection APIs are too slow to be used in the code paths targeted by these library authors, and though codegen has a large upfront cost it performs considerably better when amortized over the lifetime of the application.

This approach generally works well, but the .NET Framework is considering scenarios where it must operate in environments which do not allow codegen. This renders ineffective the existing performance improvement techniques used by these library authors.

We are uniquely positioned to provide a set of APIs which can cover the majority of scenarios traditionally involving reflection-based codegen. The general idea is that library authors can rely on the APIs we provide to work correctly both in codegen-enabled and in codegen-disallowed environments. Alternatively, the library authors can detect at runtime whether codegen is enabled, and if so they can use their existing highly-optimized codegen logic, falling back to the new API surface if codegen is disallowed.

Sample API surface

namespace System.Reflection {
    public delegate ref TField FieldAccessor<TField>(object target);

    public delegate ref TField FieldAccessor<TTarget, TField>(ref TTarget target);

    /// <summary>
    /// Provides factories that can be used by serializers, formatters, and DI systems
    /// to perform reflection-like activities in performance-critical code paths.
    /// </summary>
    [SecurityCritical]
    public static class ReflectionServices
    {
        public static bool IsCodegenAllowed { get; }

        /*
         * FIELD ACCESSORS, GETTERS, AND SETTERS
         */

        public static FieldAccessor<TField> CreateFieldAccessor<TField>(FieldInfo fieldInfo);
        public static FieldAccessor<TTarget, TField> CreateFieldAccessor<TTarget, TField>(FieldInfo fieldInfo);
        public static Func<object, object> CreateFieldGetter(FieldInfo fieldInfo);
        public static Func<object, TField> CreateFieldGetter<TField>(FieldInfo fieldInfo);
        public static Func<TTarget, TField> CreateFieldGetter<TTarget, TField>(FieldInfo fieldInfo);
        public static Action<object, object> CreateFieldSetter(FieldInfo fieldInfo);
        public static Action<object, TField> CreateFieldSetter<TField>(FieldInfo fieldInfo);
        // TTarget must not be value type; field must be an instance field.
        public static Action<TTarget, TField> CreateFieldSetter<TTarget, TField>(FieldInfo fieldInfo);
        
        /*
         * PARAMETERLESS OBJECT CREATION
         */

        public static Func<object> CreateInstanceFactory(Type type);
        public static Func<T> CreateInstanceFactory<T>();

        /*
         * PARAMETERFUL OBJECT CREATION
         */

        public static Func<object[], object> CreateInstanceFactory(ConstructorInfo constructorInfo);
        public static Func<object[], T> CreateInstanceFactory<T>(ConstructorInfo constructorInfo);
        public static Delegate CreateInstanceFactoryTyped(ConstructorInfo constructorInfo, Type delegateType);

        /*
         * PROPERTY GETTERS AND SETTERS
         * TODO: How would indexed properties be represented? Using the normal method invocation routines?
         */

        public static Func<object, object> CreatePropertyGetter(PropertyInfo propertyInfo);
        public static Func<object, TProperty> CreatePropertyGetter<TProperty>(PropertyInfo propertyInfo);
        public static Func<TTarget, TProperty> CreatePropertyGetter<TTarget, TProperty>(PropertyInfo propertyInfo);
        public static Action<object, object> CreatePropertySetter(PropertyInfo propertyInfo);
        public static Action<object, TProperty> CreatePropertySetter<TProperty>(PropertyInfo propertyInfo);
        // TTarget must not be value type; property must be an instance property.
        public static Action<TTarget, TProperty> CreatePropertySetter<TTarget, TProperty>(PropertyInfo propertyInfo);

        /*
         * METHODS
         */

        public static Func<object, object[], object> CreateMethodInvoker(MethodInfo methodInfo);
        // If instance method, 'delegateType' must be open over 'this' parameter.
        public static Delegate CreateMethodInvoker(MethodInfo methodInfo, Type delegateType);

        /*
         * EVENTS
         */
        
        public static Action<object, object> CreateEventSubscriber(EventInfo eventInfo);
        // Event must be an instance event.
        public static Action<TTarget, TDelegate> CreateEventSubscriber<TTarget, TDelegate>(EventInfo eventInfo);
        public static Action<object, object> CreateEventUnsubscriber(EventInfo eventInfo);
        // Event must be an instance event.
        public static Action<TTarget, TDelegate> CreateEventUnsubscriber<TTarget, TDelegate>(EventInfo eventInfo);
    }
}

Goals and non-goals

These APIs are not geared toward standard application developers who are already comfortable using the existing Reflection API surface. They are instead geared toward advanced library developers who need to perform Reflection operations in performance-sensitive code paths.
These APIs must work in a codegen-disallowed execution environment. (Are there exceptions?)
These APIs do not need to cover all scenarios currently allowed by the existing methods on MethodInfo and related types. For example, constructors that take ref or out parameters are sufficiently rare that we don't need to account for them. They can be invoked via the standard Reflection APIs.
These APIs do not need to have the same observable behavior as using the Reflection APIs; e.g., we may determine that these APIs should not throw TargetInvocationException on failure. But these APIs must provide consistent behavior regardless of whether they're running within a codegen-enabled or a codegen-disallowed environment.
Delegate creation does not need to be particularly optimized since there will be many checks performed upfront and we will ask callers to cache the returned delegate instances. However, once the delegates are created their invocation must be faster than calling the existing Reflection APIs. (Exception: if codegen is disallowed, then delegate invocation should be faster than calling the existing Reflection APIs wherever possible, and it must not be slower.)
It is an explicit goal to get serialization library authors to prefer this system over hand-rolling codegen for most member access scenarios. The selling points of this API would be ease of use (compared to hand-rolling codegen), performance, and the ability to work in a wide variety of execution environments.
It is an explicit non-goal to have performance characteristics equal to or better than a library's own custom codegen. For example, a DI system might choose to codegen a single method that both queries a service provider to get dependency instances and calls newobj on the target constructor. Such a system will always outperform these generalized APIs, but the API performance should be good enough that library authors would be generally satisfied using them over Reflection as a fallback in these scenarios.
These APIs do not need to support custom implementations of MemberInfo. Only support for CLR-backed members is required.

Something I forgot to mention above - there's also a consideration for making the existing Reflection APIs faster. However, this requires further thought, and there is likely only so much we can do because each invocation of the APIs would need to perform both setup and invocation. We're also not able to change the observable behavior of the existing Reflection APIs.

Sounds promising! I need to think more about how this fits all of our uses in Orleans (Serialization, RPC, Activation). This current proposal wouldn't be able to replace our existing codegen for creating proxy objects (where an interface whose methods return Task<T> is implemented via codegen to make remote calls. Currently implemented here)

Will the field setters work on initonly fields? Being able to serialize/deserialize immutable types is an important use case, especially since get-only properties are backed by initonly fields.

EDIT: currently the CLR seems to be relatively loose about how it enforces initonly since we are able to write to initonly fields from generated IL, although that IL is unverifiable. We never use IL to write to initonly fields once an object is logically constructed (deserialized / deep-copied), anyhow. A blessed path here would be greatly appreciated.

How about ref-returned property getter?

class Ref
{
    int[] _items;
    public ref int First => ref _items[0];
}

I want APIs like the following

public delegate ref TResult RefFunc<TArg, TResult>(TArg arg);

public static RefFunc<object, TProperty> CreateRefPropertyGetter<TProperty>(PropertyInfo propertyInfo);

Why don't methods like CreateMethodInvoker use generics to specify the delegate type? In similar APIs, I always find it annoying that I have to write e.g. (Action)CreateMethodInvoker(methodInfo, typeof(Action)), when it could be just CreateMethodInvoker<Action>(methodInfo).
Is CreateMethodInvoker effectively just Delegate.CreateDelegate? Though most of the other methods don't have similar equivalents, so I think it makes sense to provide it for symmetry.
For methods that use object[] for parameters, does it make sense to use custom delegate type with params instead of Func? That way, I could write e.g. CreateInstanceFactory<Foo>(constructorInfo).Invoke(3.14, "pi", null) instead of CreateInstanceFactory<Foo>(constructorInfo).Invoke(new object[] { 3.14, "pi", null }).

Unless there's a glaring technical reason why it can't be done, shouldn't there be a mass of generic overloads for CreateMethodInvoker and other methods currently showing an object[] parameter?
Otherwise there's an array creation and an arbitrary number of boxing allocations every time the provided delegate is called.

public static Func<TInstance, T1, T2, TReturn> CreateMethodInvoker<TInstance, T1, T2, TReturn>(MethodInfo methodInfo);

Useful for situations where you know a given type has a method that conforms to a signature, but it doesn't have an interface you can use to access it directly.

At least until Shapes are ready :)

The proposal so far shows an API that looks convenient to use, I like it, specially as it would remove human error from the equation.

Two question:

What makes this faster than the existing system? It is not clear to me how this API and this proposal make dynamically code generated faster.
You mention that this should work in places where no code generation is allowed, is the idea that we would run a slow code path, or that we would have an off-line code generator? The latter would probably call for a different design - see for example the way this one works: https://github.com/neuecc/Utf8Json

This is an area where I play a lot. I've been down this road many times, and have switched "engines" many times - it is very time consuming to do so. For me, frankly the "real" answer here is to get better compile-time codegen tooling - so that our libraries hook into the build chain painlessly and emit appropriate code then, without consumers needing to jump though magic hoops and arcane incantations.

In the absence of that... well, I can kinda see some benefit for greenfield scenarios, but except for the full and proper compile-time emit, personally I wouldn't feel overly compelled to try to change engine another time on an existing library.

If this is a suggestion for a new MS / corefx API: frankly I'd much rather that time was spent giving us compile-time codegen. Same target scenario, better (IMO) result.

Just my tuppence.

Additionally: unless I'm mistaken, everything here is already possible via "expression trees" - which IIRC spoof Compile() via interpreted (reflection) trees when full compilation isn't available. What would be the scenario where this new API would be a compelling improvement over expression trees?

additional additional:

The API exposed is too basic and simplistic. It isn't sufficient to just provide delegates that implement property accessors. That's enough for casual usage, but so is regular unoptimized reflection :) Given your stated audience, the typical scenario is much closer to "emit a complex single method body that accesses 12 properties on 4 inputs, performs a series of complex operations on all those things (including several custom loops), then does 3 further operations with the results - and includes exception handling". The API proposed above doesn't even begin to touch on that.

And then comes the killer word: async. Yeah, I want to be able to emit an async method that uses awaits. Oh, and it must support value-task-T as well as task-T. And support the kind of insane optimizations that library authors love, like checking task statuses to avoid an await (and the resulting state machine) when possible. Granted, these things are already incredibly hard to do via (say) ILGenerator - but are absurdly easy to do if we just emit code (whether raw C# or a Roslyn tree). The compiler is really, really good at doing this stuff. I say: let's build on the excellent compiler to solve these problems.

edit: oh, and support ref returns and ref-like types and ref-locals. No, I'm serious here. I've been in the planning stage of a new "core" for protobuf-net that uses compile-time emit, fully async, and plugs into the "channels" or "pipelines" or "streams v2" (or whatever it is called this week) API, and this is the kind of stuff that is involved.

I've built three serializers, ZeroFormatter(original format), MessagePack for C#(binary) and Utf8Json(json).

My company is creating mobile game for iOS/Android by Unity so have to support both .NET(Core) and Unity(AOT/IL2CPP).
Therefore, all serializers support two areas, runtime codegen and pre-compiled codegen.

runtime codegen

for serializer optimization, proposal api is not sufficient.
I've show decompiled code at here.
https://github.com/neuecc/Utf8Json#performance-of-serialize
for example, in serialize

// proposal design
// cost of outer accessor loop, cost of call delegate and can not avoid boxing.
foreach(var getterAccessor in accessors)
{
    writer.Write(getterAccessor.Invoke(value));
}

// Current Utf8Json design, call member directly.
writer.WriteInt32(value.foo);
writer.WriteString(value.bar);

pre-compiled codegen

In my area - Game, performance deterioration is not allowed.
Therefore, I do code generation instead of fallback to reflection.
I currently analyze .csproj(or .cs) by Roslyn and generate the source code.
I have to maintain two code bases(runtime codegen - TypeInfo, pre-compiled codegen - Roslyn), which is a burden.
I'm very happy if single codebase can do.

By the way, in my case, runtime codegen is better than pre-compiled codegen in performance.
Because x32/x64 and endianness can be determined at runtime so I generated optimized code for it but pre-compiled can not.
for example, at deserialize, embed endian dependent ulong inline.
https://github.com/neuecc/Utf8Json#performance-of-deserialize

@timcassell

@timcassell You can always use reflection.emit to make a field accessor delegate. It's just basically ldobj lfdfld ret.

@timcassell

@timcassell You can always use reflection.emit to make a field accessor delegate. It's just basically ldobj lfdfld ret.

I do believe that is not available on AOT runtimes?

If you're going AOT, honestly: "generators" would be a better option here. Do the magic during build instead.

…

On Fri, 18 Jun 2021, 07:51 Tim Cassell, ***@***.***> wrote: @timcassell <https://github.com/timcassell> You can always use reflection.emit to make a field accessor delegate. It's just basically ldobj lfdfld ret. I do believe that is not available on AOT runtimes? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23716 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEHMBZIROY3MNWGXAM3E3TTLUFJANCNFSM4OLSQUWQ> .

If you're going AOT, honestly: "generators" would be a better option here. Do the magic during build instead.

I actually want it to read from a compiler-generated state machine for an optimization (yes I know that's an implementation detail and subject to change), so generators wouldn't work for me, either.

@timcassell

@timcassell It's still being considered, it just got deprioritized compared to other work for this release.

This issue has a lot of history and still-relevant side discussions, but I think we should close this since we've been actively changing reflection internals to make it faster, and soon will be adding new APIs to continue this. The new APIs are not based on "factory" patterns although in general they are "codegen-less" unless you count internally using IL emit as codegen.

The goal for 8.0 is that existing users of IL.Emit, including internal implementations of System.Text.Json and DependencyInjection, can remove that code and use the built-in reflection APIs for fast constructors, invoke, and property\field access.

For 7.0, we increased perf 3-5x of the existing object-based Invoke() APIs when IL Emit is available. We did not change Fields, however. NativeAOT also applied similar optimizations by improving thunks and applying the same internal invocation pattern based on byref-parameter spans.

For 8.0 we plan on addressing fields and will add new APIs using byref-parameters through TypedReference for which avoids boxing (if the types are known ahead of time), supports byref-like types for invoking and passing, and avoids parameter type conversions which adds complexity and slows things down. Because the new APIs are based on byrefs, it will also support ref and out variable aliasing so an extra copy of the input parameters is not necessary to obtain the modified values. These new APIs are expected to be much faster.

Navigation Menu

Repository navigation

Proposal for high-performance codegen-less Reflection factory APIs #23716

Background

Sample API surface

Goals and non-goals

runtime codegen

pre-compiled codegen

27 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Proposal for high-performance codegen-less Reflection factory APIs #23716

Description

Background

Sample API surface

Goals and non-goals

Activity

GrabYourPitchforks commented on Oct 3, 2017

ReubenBond commented on Oct 3, 2017

ufcpp commented on Oct 3, 2017

svick commented on Oct 3, 2017

ThatRendle commented on Oct 3, 2017

migueldeicaza commented on Oct 3, 2017

mgravell commented on Oct 3, 2017

mgravell commented on Oct 3, 2017

mgravell commented on Oct 3, 2017

neuecc commented on Oct 3, 2017

runtime codegen

pre-compiled codegen

27 remaining items

En3Tho commented on Jun 18, 2021

timcassell commented on Jun 18, 2021

mgravell commented on Jun 18, 2021

timcassell commented on Jun 18, 2021

GrabYourPitchforks commented on Jun 18, 2021

steveharter commented on Oct 27, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions