Runtime async benchmarks: .NET 10 vs .NET 11

This post is part of the series 'Async/await in C#'. Be sure to check out the rest of the blog posts of the series!

• Why async exists: the cost of blocking a thread
• There is no thread: OS async I/O and the .NET thread pool
• The Task lifecycle and TaskCompletionSource
• The awaitable pattern: writing your own awaitable
• How the C# compiler rewrites an async method
• Writing your own Task from scratch
• Writing your own CancellationToken from scratch
• Async method builders and where the allocations go
• ExecutionContext: what flows across an await
• SynchronizationContext ConfigureAwait and deadlocks
• Task vs ValueTask and IValueTaskSource
• Async streams: IAsyncEnumerable and await foreach
• IAsyncDisposable and await using
• Cancellation: CancellationToken end to end
• Composing async work: WhenAll WhenAny and WhenEach
• Synchronization in async code: lock and SemaphoreSlim
• Prefer async/await over directly returning Task
• The cost of async: a BenchmarkDotNet study on .NET 10
• .NET 11 runtime async: the runtime takes over await
• Runtime async benchmarks: .NET 10 vs .NET 11 (this post)

The previous post showed what runtime async does to the generated IL. This one measures what it does to the numbers.

Read the caveats at the bottom before quoting anything here. This is a release candidate runtime, one machine, and microbenchmarks.

#How it is measured

runtime-async is an assembly wide compilation mode, so it cannot be a BenchmarkDotNet job. The benchmark is three projects compiling the same shared source:

• Net10, targeting net10.0.
• Net11Off, targeting net11.0 with <UseRuntimeAsync>false</UseRuntimeAsync>.
• Net11On, targeting net11.0 with <Features>runtime-async=on</Features>.

Net11Off is what separates "the new runtime is different" from "runtime async is different", and it turns out to be essential.

Only the project files differ. Net11On opts in, Net11Off opts out, and Net10 predates the feature:

<!-- Net11On -->
<TargetFramework>net11.0</TargetFramework>
<Features>$(Features);runtime-async=on</Features>

<!-- Net11Off -->
<TargetFramework>net11.0</TargetFramework>
<UseRuntimeAsync>false</UseRuntimeAsync>

<!-- Net10 -->
<TargetFramework>net10.0</TargetFramework>

All three compile the same shared workloads, so any difference in the numbers comes from how they were compiled and nothing else:

// Never suspends. This is the path where the state machine cost is the whole cost.
public static async Task<int> CompletedAsync(int depth)
    => depth == 0 ? 42 : await CompletedAsync(depth - 1);

// Suspends at the bottom of the chain.
public static async Task<int> SuspendingAsync(int depth)
{
    if (depth == 0)
    {
        await Task.Yield();
        return 42;
    }

    return await SuspendingAsync(depth - 1);
}

// Locals that outlive the await, which the compiler has to hoist and the runtime does not.
public static async Task<int> WithLocalsAsync()
{
    var a = Guid.NewGuid();
    var b = a.ToString();
    var c = b.Length;

    await Task.Yield();
    return a.GetHashCode() + b.Length + c;
}

public static async IAsyncEnumerable<int> StreamAsync(int count)
{
    for (var i = 0; i < count; i++)
    {
        await Task.CompletedTask;
        yield return i;
    }
}

[InProcess]
[MemoryDiagnoser]
public class AsyncBenchmarks
{
    [Benchmark]
    public int CompletedDepth1() => Workloads.CompletedAsync(1).GetAwaiter().GetResult();

    [Benchmark]
    public int CompletedDepth20() => Workloads.CompletedAsync(20).GetAwaiter().GetResult();

    [Benchmark]
    public int SuspendingDepth1() => Workloads.SuspendingAsync(1).GetAwaiter().GetResult();

    [Benchmark]
    public int SuspendingDepth20() => Workloads.SuspendingAsync(20).GetAwaiter().GetResult();

    [Benchmark]
    public int WithLocals() => Workloads.WithLocalsAsync().GetAwaiter().GetResult();

    [Benchmark]
    public int AsyncStream() => StreamAsync().GetAwaiter().GetResult();

    private static async Task<int> StreamAsync()
    {
        var total = 0;
        await foreach (var item in Workloads.StreamAsync(1000))
        {
            total += item;
        }

        return total;
    }
}

#The results

#The headline: synchronous chains collapse

Look at CompletedDepth20. Twenty nested async methods, none of which ever suspends:

• .NET 10: 155 ns, 1512 bytes
• .NET 11 with runtime async: 44 ns, 72 bytes

Three and a half times faster, and twenty one times less memory. The 1512 bytes was twenty Task<int> allocations, one per layer. The 72 bytes is one.

This is the optimisation that a compiler cannot do and a runtime can. Each layer is now an ordinary method the JIT can see through, so a call chain that never actually suspends is inlined away, and only the outermost method needs a Task to hand back. Depth stopped costing anything: CompletedDepth1 and CompletedDepth20 both allocate 72 bytes.

That matters because deep pass-through chains are what real applications are made of. A controller calling a service calling a repository calling a cached client, where the cache hits, is exactly this shape. It is also the case where the previous post in this series recommended ValueTask; runtime async gets much of that benefit without the rules.

Even at depth 1 the allocation halves, from 144 bytes to 72.

The middle column makes the win look even larger. On these synchronous paths Net11Off is about 30% slower than .NET 10: 18 ns against 13 ns at depth 1, 205 ns against 155 ns at depth 20, with identical allocations. Measured against .NET 11 without the feature, runtime async makes the 20 layer chain more than four times faster.

#Suspension: deep chains win, shallow ones allocate more

SuspendingDepth20 allocates 1472 bytes with runtime async against 1980 on .NET 10, about 26% less, and runs about 13% faster: 1862 ns against 2146 ns. The runtime reuses continuation objects and avoids saving locals that have not changed, and a deep chain is where that adds up.

Shallow suspensions are the numbers that go the wrong way. SuspendingDepth1 allocates 256 bytes against 206, and WithLocals 288 against 216, while Net11Off matches .NET 10 byte for byte on both. So the extra bytes come from runtime async itself: when only one or two frames suspend, the continuations the runtime allocates cost more than the state machine box they replace. WithLocals is the counterintuitive one, because hoisting locals is precisely what the compiler no longer does; the runtime still has to save them somewhere when the method suspends.

The extra memory does not cost time. SuspendingDepth1 is about 17% faster than on .NET 10, and WithLocals is within noise. It is a few dozen bytes per suspension, on a path that already costs over a microsecond.

#Async streams

AsyncStream iterates 1000 items that are all already available. Runtime async takes it from 6043 ns to 4786 ns, a 20% improvement, with identical allocations.

That fits the pattern: the win is on the synchronous path, where the generated machinery is the entire cost. An async iterator's MoveNextAsync completes synchronously for almost every item, so a thousand items are a thousand trips through the fast path.

#What this means in practice

If the numbers hold at release:

Deep async chains stop being a reason to worry. The main structural cost of async in .NET 10, one Task per layer, largely disappears when the layers do not suspend.

ValueTask becomes less necessary. Not useless: a method returning a value it already has still avoids an allocation entirely. But the "wrap it in ValueTask because the chain is deep" argument weakens considerably, and given the rules ValueTask imposes, that is a good trade.

The pooling builder becomes even more niche. It was already saving a few percent on the suspending path. Runtime async does its own continuation reuse.

Suspending is still expensive. Nothing here changes the shape from the .NET 10 cost study: the synchronous path is nanoseconds and suspending is microseconds. Runtime async makes the cheap path cheaper. It does not make suspension free, and no runtime can.

#Caveats

Stated plainly, because these numbers will be quoted:

• Release candidate 3, not the final release. The numbers can still move. On Preview 7 every suspending benchmark was about four times slower on .NET 11, with and without runtime async; that regression is gone in RC3.
• One machine, one architecture. Apple M5 Pro, arm64, macOS. Thread pool behaviour in particular differs across platforms.
• Microbenchmarks. Task.Yield is not a network call. These measure the machinery, deliberately, and the machinery is a small part of a real workload.
• The Net11Off slowdown on synchronous paths is unexplained. It is a runtime difference, not a runtime async one, and it is the reason to keep the middle column.

Re-run the benchmark yourself before making a decision; the project is in the download.

#What to take away

• Runtime async collapses a 20 layer chain that never suspends from 1512 bytes to 72, and from 155 ns to 44 ns.
• Depth stopped costing allocations on the synchronous path.
• A single-layer synchronous async call halves, 144 bytes to 72.
• A deep suspending chain allocates about 26% less and runs about 13% faster, thanks to continuation reuse.
• A shallow suspension allocates a few dozen bytes more than on .NET 10, without being slower.
• Async streams over already-available items get about 20% faster.
• Release candidate, one machine, in-process toolchain. Measure your own workload.

The last post of the series is about what all this means for code you already have: what changes, what does not, and what to check before turning it on.

#Additional resources

• Benchmark
• What's new in the .NET 11 runtime
• links about async/await in C#

Do you have a question or a suggestion about this post? Contact me!