Composing async work: WhenAll WhenAny and WhenEach

This post is part of the series 'Async/await in C#'. Be sure to check out the rest of the blog posts of the series!

• Why async exists: the cost of blocking a thread
• There is no thread: OS async I/O and the .NET thread pool
• The Task lifecycle and TaskCompletionSource
• The awaitable pattern: writing your own awaitable
• How the C# compiler rewrites an async method
• Writing your own Task from scratch
• Writing your own CancellationToken from scratch
• Async method builders and where the allocations go
• ExecutionContext: what flows across an await
• SynchronizationContext ConfigureAwait and deadlocks
• Task vs ValueTask and IValueTaskSource
• Async streams: IAsyncEnumerable and await foreach
• IAsyncDisposable and await using
• Cancellation: CancellationToken end to end
• Composing async work: WhenAll WhenAny and WhenEach (this post)

Everything so far has been about a single operation. This post is about running several, which is where async stops being a syntax feature and starts changing what your program does.

The mechanics are three combinators and one throttle, and each has an exception behaviour that is worth knowing before you meet it in production.

#Starting is what makes them concurrent

Before the combinators, the thing they depend on. An async method starts running when you call it, not when you await it. That is what makes concurrency possible:

// Concurrent: every operation is started, then all are awaited.
var tasks = urls.Select(url => FetchAsync(url)).ToArray();
var results = await Task.WhenAll(tasks);

// Sequential: each is started and finished before the next begins.
foreach (var url in urls)
{
    results.Add(await FetchAsync(url));
}

The ToArray() is load bearing. Select is lazy, so without it the sequence is enumerated inside WhenAll, and although that still works it is a habit worth keeping: materialise, then await.

The benchmark, 100 operations of 5 ms each:

A hundredfold improvement, for less than a kilobyte more. That is the case for concurrency in one line.

The throttled versions take 13 times longer than WhenAll, which is exactly right: they allow 8 at a time rather than 100, so they take 100/8 rounds. You do not pay that for nothing, and the section on throttling explains what you buy.

#WhenAll

Waits for everything. The exception behaviour is the part to internalise:

var task = Task.WhenAll(
    Task.FromException(new InvalidOperationException("first")),
    Task.FromException(new ArgumentException("second")));

// await rethrows only the first
await Assert.ThrowsAsync<InvalidOperationException>(() => task);

// but all of them are on the task
Assert.Equal(2, task.Exception.InnerExceptions.Count);

Every task runs to completion, even after one fails. WhenAll does not cancel the others; it has no way to.

All exceptions are collected, but await unwraps to the first. If you need them all, keep the task and read task.Exception:

var task = Task.WhenAll(tasks);
try
{
    await task;
}
catch
{
    foreach (var exception in task.Exception!.InnerExceptions)
    {
        logger.LogError(exception, "Operation failed");
    }

    throw;
}

This is the same trap as getting all exceptions from Parallel.ForEachAsync, and it is why "only one of my five failures was logged" is such a common report.

#WhenAny

Returns as soon as one task finishes. Useful for "first response wins" and for racing an operation against a timeout, and it has a sharp edge:

var winner = await Task.WhenAny(fast, slowAndFailing);
// slowAndFailing is still running, and its exception is unobserved

The losers keep running. They hold their connections, their memory and their thread pool work items, and nobody is watching their results. If a loser fails, the exception goes nowhere.

So WhenAny needs a cleanup plan:

• Pass a CancellationToken to every operation and cancel it once you have a winner. This is the good answer.
• Or at minimum observe the losers so their failures are logged rather than silently dropped.

Since .NET 6, racing against a timeout does not need WhenAny at all. WaitAsync says it more directly:

// Instead of Task.WhenAny(work, Task.Delay(timeout))
await work.WaitAsync(TimeSpan.FromSeconds(5), cancellationToken);

#WhenEach

Added in .NET 9, and it fills a real gap. Before it, processing results as they arrived meant a loop over WhenAny with a list you removed from, which is O(n²) and easy to get wrong:

await foreach (var task in Task.WhenEach(tasks))
{
    var result = await task;
    Process(result);
}

Tasks come out in completion order, not in the order you passed them. The sample checks that a fast operation is yielded before a slow one started earlier.

The inner await task is not redundant. WhenEach hands you the completed Task, and awaiting it is what produces the value or throws its exception. That also means you can handle each failure individually and keep going, which neither WhenAll nor WhenAny lets you do.

Use it when results are worth processing as they arrive rather than in a batch at the end: streaming to a client, updating a UI, or writing to a pipeline.

#Throttling

WhenAll over ten thousand items starts ten thousand operations at once. Your database will notice. Two ways to bound it:

Parallel.ForEachAsync, which is the right default:

await Parallel.ForEachAsync(items, new ParallelOptions
{
    MaxDegreeOfParallelism = 8,
    CancellationToken = cancellationToken,
}, async (item, cancellationToken) => await ProcessAsync(item, cancellationToken));

It handles the concurrency limit, threads the token through, and works on an IAsyncEnumerable<T> as well as an IEnumerable<T>, so you can throttle a stream without materialising it.

SemaphoreSlim, when you need the limit shared across code that is not one loop:

await semaphore.WaitAsync(cancellationToken);
try
{
    await ProcessAsync(item, cancellationToken);
}
finally
{
    semaphore.Release();
}

The finally is mandatory. A Release that is skipped on an exception path permanently reduces the limit, and a few of those turn into a hang that looks like a deadlock.

In the benchmark the two are indistinguishable in time, with the semaphore allocating about 30% more. Prefer Parallel.ForEachAsync unless you need a semaphore's flexibility.

#What to take away

• Concurrency comes from starting the operations, not from the combinator. Materialise the sequence first.
• WhenAll runs everything even after a failure and collects all exceptions, but await rethrows only the first. Read task.Exception when you need them all.
• WhenAny leaves the losers running and their exceptions unobserved. Cancel them, or observe them.
• Use WaitAsync rather than racing against Task.Delay.
• WhenEach (.NET 9) yields tasks in completion order and lets you handle each failure independently.
• Throttle with Parallel.ForEachAsync. Use SemaphoreSlim when the limit spans more than one loop, and release in a finally.

The next post covers what happens when concurrent operations need to touch the same state: locking in async code, and why lock and await do not mix.

#Additional resources

• Code sample
• Benchmark
• Get the result of multiple tasks in a ValueTuple and WhenAll

Do you have a question or a suggestion about this post? Contact me!