Async streams: IAsyncEnumerable and await foreach

This post is part of the series 'Async/await in C#'. Be sure to check out the rest of the blog posts of the series!

• Why async exists: the cost of blocking a thread
• There is no thread: OS async I/O and the .NET thread pool
• The Task lifecycle and TaskCompletionSource
• The awaitable pattern: writing your own awaitable
• How the C# compiler rewrites an async method
• Writing your own Task from scratch
• Writing your own CancellationToken from scratch
• Async method builders and where the allocations go
• ExecutionContext: what flows across an await
• SynchronizationContext ConfigureAwait and deadlocks
• Task vs ValueTask and IValueTaskSource
• Async streams: IAsyncEnumerable and await foreach (this post)

An async method produces one value. An iterator produces many values but cannot await. IAsyncEnumerable<T> is what happens when you need both, and the compiler builds it by fusing the two state machines it already knows how to generate.

The typical case is paging through an API: each page is a network call, and the caller should not have to know that pages exist.

await foreach (var item in api.GetItemsAsync())
{
    Process(item);
}

#The interfaces

Three of them, and they mirror the synchronous ones exactly:

public interface IAsyncEnumerable<out T>
{
    IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default);
}

public interface IAsyncEnumerator<out T> : IAsyncDisposable
{
    T Current { get; }
    ValueTask<bool> MoveNextAsync();
}

Two differences from IEnumerator<T> are worth pausing on.

MoveNextAsync returns ValueTask<bool>, not Task<bool>. This is exactly the case ValueTask was designed for. Most items are already buffered, so most calls complete synchronously. A Task<bool> per item would allocate on every element of every loop. The sample's tests check that the second MoveNextAsync of a fetched page is IsCompletedSuccessfully.

GetAsyncEnumerator takes a CancellationToken. The synchronous version has nowhere to put one. This is the hook WithCancellation uses.

IAsyncEnumerator<T> deriving from IAsyncDisposable is what lets the compiler clean up when you leave the loop early, which is the next post's subject.

#Writing one

async plus yield return, and the return type is IAsyncEnumerable<T>:

public static async IAsyncEnumerable<int> GetItemsAsync(int pageCount, int pageSize, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    for (var page = 0; page < pageCount; page++)
    {
        var items = await FetchPageAsync(page, pageSize, cancellationToken);
        foreach (var item in items)
        {
            yield return item;
        }
    }
}

The generated type implements IAsyncEnumerable<T>, IAsyncEnumerator<T>, IAsyncStateMachine and IValueTaskSource<bool> all at once, and reuses one object for the whole enumeration. It combines the <>1__state field of an async state machine with the resume-position tracking of an iterator: one state number covers both "which await did I stop at" and "which yield return did I stop at".

Two behaviours follow from that, and they surprise people coming from async Task:

Nothing runs until you enumerate. Calling the method returns the object without executing a line of the body. An async Task method runs eagerly up to its first suspending await; an async iterator does not run at all until the first MoveNextAsync.

Each enumeration is independent. GetAsyncEnumerator hands out fresh state, so enumerating twice runs the body twice, making two sets of network calls.

#The attribute everybody forgets

This is the single most common bug with async streams:

await foreach (var item in GetItemsAsync().WithCancellation(token))

WithCancellation passes the token to GetAsyncEnumerator. But your method signature has its own CancellationToken parameter, and nothing connects the two unless you say so:

public static async IAsyncEnumerable<int> GetItemsAsync([EnumeratorCancellation] CancellationToken cancellationToken = default)

[EnumeratorCancellation] tells the compiler which parameter should receive the token from GetAsyncEnumerator. Without it, WithCancellation compiles, runs, and silently does nothing: the parameter keeps its default value and the loop cannot be cancelled.

#What it costs

Consuming 1000 items that are all already available:

Eight times slower than a plain foreach, for 1000 items, when every item is already in memory. That sounds bad and mostly is not: it is six microseconds for a thousand items, roughly 6 ns each, and if any of those items had involved a real network call the transport would have cost a thousand times more.

The number that matters is the allocation column: 184 bytes total, not per item. The single fused state machine object serves the whole enumeration. That is the design working.

Channel<T> looks terrible here at 19 KB, and that comparison is unfair to it. A channel is a queue between a producer and a consumer running concurrently; this benchmark uses it as a list, which is the one thing it is not for. Use a channel when producer and consumer are genuinely separate and you need buffering or backpressure; use an async stream when one side pulls from the other.

ToBlockingEnumerable is the escape hatch for consuming an async stream from synchronous code. It costs about 13% more than the async loop here, but that is not the real price: it blocks a thread for the whole enumeration, with all the consequences from the first post. Use it at the boundary of legacy code, not as a convenience.

#ConfigureAwait on a loop

await foreach awaits MoveNextAsync and DisposeAsync for you, so there is nowhere to put a ConfigureAwait(false). The extension method goes on the enumerable:

await foreach (var item in stream.ConfigureAwait(false))

And when you need both, they chain in this order:

await foreach (var item in stream.WithCancellation(token).ConfigureAwait(false))

WithCancellation returns a ConfiguredCancelableAsyncEnumerable<T> that also has ConfigureAwait, which is why that order works and the reverse does not.

#What to take away

• IAsyncEnumerable<T> fuses an async state machine with an iterator into a single generated object.
• MoveNextAsync returns ValueTask<bool> because most items are buffered and complete synchronously.
• Nothing runs until you start enumerating, and each enumeration runs the body again.
• [EnumeratorCancellation] is required for WithCancellation to reach your parameter. Without it, cancellation silently does nothing.
• The whole enumeration allocates once, not once per item.
• ConfigureAwait and WithCancellation go on the enumerable, and chain in that order.
• Reach for Channel<T> when producer and consumer run concurrently, not as a replacement for a stream.

The next post covers the other half of IAsyncEnumerator: IAsyncDisposable, await using, and what the compiler generates when you leave a loop early.

#Additional resources

• Code sample
• Benchmark
• Using IAsyncEnumerable in a Razor component
• links about async/await in C#

Do you have a question or a suggestion about this post? Contact me!