Developers using reflection invoke should be able to use ref struct

Support ref struct and "fast invoke" by adding new reflection APIs. The new APIs will be faster than the current object[] boxing approach by leveraging the existing System.TypedReference<T>.

TypedReference is a special type and is super-fast because since it is a ref struct with its own opcodes. By extending it with this feature, it provides alloc-free, stack-based “boxing” with support for all argument types (reference types, value types, pointers and ref structs [pending]) along with all modifiers (byval, in, out, ref, ref return). Currently reflection does not support passing or invoking a ref struct since it can’t be boxed to object; the new APIs are to support ref struct with new language features currently being investigated.

Example syntax (actual TBD):

MyClass obj= …; 
MethodInfo methodInfo = …;

int param1 = 42; // Pass an integer
Span<int> param2 = new Span<int>(new int[]{1, 2, 3}); // Pass a span (not possible with reflection today)

// Adding support for Span<TypedReference> requires the language and runtime asks below.
Span<TypedReference> parameters = stackalloc TypedReference[2];

// The 'byval', 'in', 'out' and 'ref' modifiers in the callee method's parameters all have the same caller syntax
parameters[0] = TypedReference.FromRef(ref param1);
parameters[1] = TypedReference.FromRef(ref param2);
methodInfo.GetInvoker().Invoke(returnValue: default, target: TypedReference.FromRef(ref obj), parameters);  
// The first call to this methodInfo will be slower; subsequent calls used cached IL when possible

// Shorter syntax using __makeref (C# specific)
parameters[0] = __makeref(param1);
parameters[1] = __makeref(param2);
methodInfo.Invoke(default, __makeref(obj), parameters));

Dependencies

The Roslyn and runtime dependencies below are required for the programming model above. These are listed in the order in which they need to be implemented.

ref fields: necessary to rationalize TypedReference.
- Allow ref struct to contain ref fields #32060
- ref field support in .NET runtimes #63768
Support passing a ref struct as a generic argument. This would be used when adding the static factory method public static TypedReference CreateFromRefStruct<T>(ref T myRefStruct) where T : ref struct which basically wraps __makeref(myrefstruct). It could also be used to enable Span<T> where T : ref struct which is the ideal stack-based collection implementation that can contain TypedReferences.
An arbitrary array/span of references to enable a stack-based, variable length container for method arguments. See also related comments in ref fields: necessary to rationalize TypedReference and prototype using params. The new invoke APIs require a collection of TypeReferences. There are several possible implementations; the most normalized solution would be supporting Span<T> where T : ref struct thus enabling any ref struct (not just TypedReference) to be used in a container. UPDATE: done in prototype per [API Proposal]: byref parameter collection for invoke #75349
Removing the concept of restricted types including TypedReference. (Roslyn link TBD). Ideally, TypedReference is a normal ref struct. Note that TypedReference currently has ByReference<byte> to contain an interior pointer to the value so this will likely need a different type. Also, TypedReference has several compile-time limitations including not be able to be passed to another method that should be removed resulting in only standard ref struct semantics.
Add reflection support for Span and other ref struct types (this is the library\API issue and requires all of the above work items)
- See disabled test for ref fields.

Motivation

Reflection is ~20x slower than a Delegate call for a typical method. Many users including our own libraries use IL Emit instead which is non-trivial and error-prone. The expected gains are ~10x faster with no allocs; verified with a prototype. Internally, IL Emit is used but with a proposed slow-path fallback for AOT (non-emit) cases. The existing reflection invoke APIs may also layer on this.

In Scope 

APIs to invoke methods using TypedReference including passing a TypedReference collection. TypedReference must be treated as a normal ref struct (today it has nuances and special cases).

Support ref struct (passing and invoking).

Performance on par with existing ref emit scenarios:

Property get\set
Field get\set
Parameterized constructors and methods

To scope this feature, the minimum functionality that results in a win by allowing System.Text.Json to remove its dependency to System.Reflection.Emit for inbox scenarios.

Out of Scope 

This issue is an incremental improvement of reflection by adding new Invoke APIs and leveraging the existing TypedReference while requiring some runtime\Roslyn changes. Longer-term we should consider a more holistic runtime and Roslin support for reflection including JIT intrinsics and\or new "dynamic invoke" opcodes for performance along with perhaps C# auto-stack-boxing to\from a ref TypedReference.

Implementation

A design doc is forthcoming.

The implementation will likely cache the generated method on the corresponding MethodBase and MemberInfo objects.

100% backwards compat with the existing object[]-based Invoke APIs is not necessary but will be designed with laying in mind (e.g. parameter validation, special types like ReflectionPointer, the Binder pattern, CultureInfo for culture-aware methods) so that in theory the existing object[]-based Invoke APIs could layer on this new work.

This issue supersedes other reflection performance issues that overlap:

@pranavkm

This isn't just for JSON, what we've we doing should work for any framework that uses reflection today I assume. I'm hoping to see a deep analysis here of the scenarios that Would use these APIs

cc @pranavkm @SteveSandersonMS

PowerShell could be benefit from this - it is based on reflection, emit and dynamic.

As mentioned in some of the previous issues, and to add a note on the requirements as there is a reference to Reflection.Emit that might make this not clear: the solution should not involve any managed objects from System.Reflection but go through only RuntimeXXXHandle that can work directly with ldtoken&co.

We have discussed that these APIs should be efficient and avoid allocating lots of managed memory: #28001 (comment) . Avoiding any managed objects from System.Reflection is much more stringent and unlikely to be practical.

Avoiding any managed objects from System.Reflection is much more stringent and unlikely to be practical.

Fair enough. I don't know the cases that would involve reflection, but as long as simple cases like accessing statically known e.g fields directly from IL via ldtoken, are working without, that's ok.

@jkotas

How much work is this? Is there any way to make incremental progress in 6.0 towards it?

I'm writing up something on the code generation techniques used in ASP.NET Core and for the future reference I'd like an idea of how our APIs would need to evolve. The closest I've found is @jkotas 's comment here #10057 (comment).

We may be able to make some incremental progress on faster reflection in .NET by refactoring the internal implementation.

We won't be able to expose the new faster reflection APIs for .NET 6. For that, we need #26186 that depends on https://github.com/dotnet/csharplang/blob/master/proposals/low-level-struct-improvements.md that was cut for .NET 6.

Got it. For .NET 7 then. I think I understand how to model the APIs we need to take advantage of this. Unfortunately this and async are at odds, but we can probably optimize the sync path with this and box in the async path. Still that would mean we need a way to create a TypedReference from this boxed value in a way that won't explode because the new API won't handle type coercion i.e a mix of this API needs to support a mix of boxed and unboxed values somehow. I can provide examples if needed.

@jaredpar

Bumping again on the topic, more precisely for the support of offsetof and so by extension handleof(A.MyField) that we discussed here), can we add here that it might require a language + Roslyn support, so that we can build higher primitives/intrinsics around it? (e.g like Unsafe.OffsetOf(fieldToken)).

A scenario example for offsetof is when building for example an animation system that can animate dynamic (resolution based on data, not code) field values of a struct that is stored in an unmanaged memory buffer.

cc: @jaredpar (for the question about Language + Roslyn support)

I'm not sure how the language can implement offsetof without some underlying runtime support as it's a value you can't determine until runtime. As for that and handleof I think it comes down to if there is enough benefit in adding it, particularly when it's just a think wrapper over .NET API calls.

Just tested and calling the following code is working with .NET 7 JIT, but I haven't checked the specs in a while for such operation.

This is invalid IL. If you run it on checked JIT, you will see all sorts of asserts. The first one is:

Assertion failed 'genActualType(op1->TypeGet()) != TYP_REF && genActualType(op2->TypeGet()) != TYP_REF : Possibly bad IL with CEE_add at offset 0002h (op1=ref op2=long stkDepth=0)' in 'Test:AddByteOffset[ubyte](System.Object,long):byref' during 'Importation' (IL size 4; hash 0xcdd16edf; Tier0)

It is likely that that the JIT can either crash or produce bad code when this gets method inlined into a callsite of a particular shape.

This is invalid IL. If you run it on checked JIT, you will see all sorts of asserts. The first one is:

Fair enough. We definitely don't want invalid IL 😅

So, exposing RuntimeHelpers.GetRawData would be the way and it would require to shift the offset with the method table in the UnsafeAccessor code in my PR, or could we provide a ref byte Unsafe.AsByteRef(object) { ldarg.0; ret; }?

I'm ok with RuntimeHelpers.GetRawData if it is the preferred way.

My opinion is that the following set of APIs makes the most sense:

namespace System.Runtime.CompilerServices
{
    public static class RuntimeHelpers
    {
        public static ref byte GetRawData(object? o);
        public static object? GetObject(ref byte reference, out nuint offset);
    }
}

This provides the forward and reverse API to convert between byrefs and objects.

The lack of <T> reduces the number of generic instantiations we will get (if that still matters on some platforms? I can't recall).

Note object?: since we will need to check null anyway, I think it makes sense to check for null and give a null byref for GetRawData, instead of throwing, since it should be able to emit better code for that, and for GetObject it makes sense since reference could point to unmanaged/stack memory (in which case offset would equal the pointer).

When the documentation is written for these pair of APIs, we will need to consider their behaviour with strings and arrays - currently they would probably return a ref to the start of the length field - if this is how we want it to work (which probably makes sense), then we should document the offset from the length field to the first entry in these cases so people can use it correctly for these OR we could document that it's undefined to use these APIs and then try to determine/select what specific index it's at (this would allow us to change them to be 64 bit length in the future if needed), which could make sense since there are other APIs for working with these special cases in the "intended" way when indexing is desired.

Linking Roslyn issue dotnet/roslyn#68000 where you can't get an offset of a ref field.

We now have the PR at #93946 which adds UnsafeAccessorKind.FieldOffset (API issue pending).

But we need a champion to create the API issue for the RuntimeHelpers.GetRawData() and .GetObject() proposal above so we can move the discussion there.

But we need a champion to create the API issue for the RuntimeHelpers.GetRawData()

I simply reopened #28001 since it already has a bunch of discussion and it's exactly that proposal. I cleared the milestone and marked untriaged.

It'd be very nice if the System.TypedReference modernization can happen for 10.0, all the pieces should be in place. For the time being I'm using my own version.

readonly ref struct TypedReference
{
    readonly ref byte _value;
    public Type Type { get; }

    TypedReference(ref byte target, Type type)
    {
        _value = ref target;
        Type = type;
    }

    public ref T UnsafeAsRef<T>() => ref Unsafe.As<byte, T>(ref _value);
    public ref T AsRef<T>()
    {
        if (typeof(T) != Type)
            ThrowInvalid();

        return ref Unsafe.As<byte, T>(ref _value);
    }
    static void ThrowInvalid() => throw new InvalidCastException("Invalid type given.");

    public static TypedReference Create<T>(ref T value) => new(ref Unsafe.As<T, byte>(ref value), typeof(T));
}

Within controlled and easily verifiable areas I have also used an UnsafeReference variant. This is essentially just a ref byte ref struct with some validation baked in for debug (though that won't really help if the BCL would ship that to consumers).

Maybe somebody finds this useful:

#if DEBUG
readonly ref struct UnsafeReference
{
    readonly ref byte _value;
    readonly Type _type;

    UnsafeReference(ref byte target, Type type)
    {
        _value = ref target;
        _type = type;
    }

    public ref T UnsafeAsRef<T>() => ref Unsafe.As<byte, T>(ref _value);
    public ref T AsRef<T>()
    {
        Debug.Assert(typeof(T) == _type);
        return ref Unsafe.As<byte, T>(ref _value);
    }

    public static implicit operator UnsafeReference(TypedReference typedReference) => new(ref typedReference.UnsafeAsRef<byte>(), typedReference.Type);
    public static UnsafeReference Create<T>(ref T value) => new(ref Unsafe.As<T, byte>(ref value), typeof(T));
}
#else
readonly ref struct UnsafeReference
{
    readonly ref byte _value;

    UnsafeReference(ref byte target)
    {
        _value = ref target;
    }

    public ref T UnsafeAsRef<T>() => ref Unsafe.As<byte, T>(ref _value);
    public ref T AsRef<T>() => ref UnsafeAsRef<T>();

    public static implicit operator UnsafeReference(TypedReference typedReference) => new(ref typedReference.UnsafeAsRef<byte>());
    public static UnsafeReference Create<T>(ref T value) => new(ref Unsafe.As<T, byte>(ref value));
}
#endif

It'd be very nice if the System.TypedReference modernization can happen for 10.0, all the pieces should be in place.

The pieces are not all in place yet. TypedReference is effectively an opaque storage for a ref value of any type. That includes ref struct and our lifetime model does not support that yet. There is a proposal that will close this gap enough that we could correctly model TypedReference in the type system. It would require unsafe when unpacking an instance and getting a value out by ref but otherwise would be safe.

It'd be very nice if the System.TypedReference modernization can happen for 10.0, a

That seems unlikely at this point. The language work is mostly understood but this is tied to runtime changes. At the moment I don't think the combined runtime + language work will fit into the .NET 10 schedule. Think 11 is more likely.

It would require unsafe when unpacking an instance and getting a value out by ref but otherwise would be safe.

This would be a really unfortunate outcome. As a user of a type like this I would much prefer to have two operations to confer it is specifically the ref struct types that need user diligence to be used safely. Concretely I would prefer it to be one operation for unsafe extraction - with T : allow ref struct - and another safe one without, doing the safe fallible runtime cast that it resembles. Allowing it to be used 100% safely for most types avoids unnecessary unsafe proliferation in the ecosystem.

I previously left a good reason for wanting this type to be usable in safe context in dotnet/roslyn#65255 (comment)
It's a fantastic mechanism for avoiding generic specialization bloat of internal code. Where there is a sizable non-generic spine of code acting on a generic value that - if not for wanting to keep structs unboxed - would just take a callback and object? state (or more commonly won't be modeled to allow for structs at all, opting to refer to some abstract base class instead).

The bloat incurred in those cases is a fundamental problem for using structs in 'open world' api designs. Passing TypedReferences through the internals instead can make structs viable in *all* dimensions for high perf code, only requiring a cheap indirection to isolate the generic call from the non-generic code. These advantages of structs won't diminish with better PGO or more on stack allocation of classes and so on. Structs guarantee most of their advantages in overhead and predictability, for NativeAOT, crossing abstractions, in tier 0 and so on, which is a huge advantage in itself.

The pieces are not all in place yet. TypedReference is effectively an opaque storage for a ref value of any type. That includes ref struct and our lifetime model does not support that yet. There is a proposal that will close this gap enough that we could correctly model TypedReference in the type system

var span = Span<byte>.Empty;
var typedReference = __makeref(span); // error CS1601

The above doesn't work today, including in an unsafe context, so it's currently not usable for any type anyway. It would be fantastic to remove that limitation, but that same reasoning holds for Spans, of either form (though safely only ReadOnlySpan applies as I understand it). All could benefit, but there is already lots of value in supporting the current safe set of types (Span's baseline usefulness being much higher of course).

The modernized version of TypedReference should allow for the future addition of T : allow ref struct on the relevant apis, but that being a supported scenario - and an evident future need - I still can't see what isn't in place to do any modernization today. I fully understand that doing the design and changes in two phases could require more investment than the type warrants but that's not clear to me at this point, so please bear with me.

It would require unsafe when unpacking an instance and getting a value out by ref but otherwise would be safe.

This would be a really unfortunate outcome ... Concretely I would prefer it to be one operation for unsafe extraction - with T : allow ref struct - and another safe one without, doing the safe fallible runtime cast that it resembles.

Sorry, let me clarify. Extracting a value by ref T from a TypedReference would be unsafe if T is potentially a ref struct. Otherwise, it would be safe. The rational for ref struct case being unsafe is that safely representing that is not possible without adding explicit lifetimes to the language where as today C# works primarily by inferring lifetimes.

The above doesn't work today, including in an unsafe context, so it's currently not usable for any type anyway.

That is by design. We didn't want to allow ref struct to be usable with __makeref until we had a plan on how to handle the lifetimes. If we adopt the proposal I listed then we would remove this restriction.

Navigation Menu

Repository navigation

Developers using reflection invoke should be able to use ref struct #45152

Dependencies

Motivation

In Scope

Out of Scope

Implementation

72 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Developers using reflection invoke should be able to use ref struct #45152

Description

Dependencies

Motivation

In Scope

Out of Scope

Implementation

Activity

davidfowl commented on Nov 24, 2020

iSazonov commented on Nov 24, 2020

xoofx commented on Nov 24, 2020

jkotas commented on Nov 24, 2020

xoofx commented on Nov 24, 2020

davidfowl commented on May 16, 2021

jkotas commented on May 16, 2021

davidfowl commented on May 16, 2021

xoofx commented on May 26, 2021

jaredpar commented on May 26, 2021

72 remaining items

jkotas commented on Oct 25, 2023

xoofx commented on Oct 25, 2023

hamarb123 commented on Oct 25, 2023

steveharter commented on Oct 26, 2023

steveharter commented on Oct 26, 2023

MichalStrehovsky commented on Oct 27, 2023

NinoFloris commented on Jan 17, 2025

jaredpar commented on Jan 21, 2025

NinoFloris commented on Jan 22, 2025

jaredpar commented on Jan 22, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

In Scope 

Out of Scope 