Inside async/await in C#

It’s been a long time since C# provided the async/await feature to help with potentially long-running tasks. The keywords async and await first appeared in C# 5.0, released in 2012 together with .NET Framework 4.5 and Visual Studio 2012. The feature was introduced to (greatly) simplify asynchronous programming that, while available in .NET since day 1, always relied for implementation on a variety of byzantine and tricky programming patterns.

The Time Before

In the very beginning, it was the BeginXxx / EndXxx pair of methods which were supplanted by the Task Parallel Library (TPL) in 2010 along with the advent of .NET 4.0. Although designed with the best intentions, TPL still required a lot of effort and risk to developers. This resulting code quickly became difficult to read because any logic based on TPL ended up fragmented across nested continuation steps. In fact, by design, TPL asynchronous code using the Task class required explicit continuations. Here’s an example:

GetDataAsync()
  .ContinueWith(t =>
  {
     var data = t.Result;
     Process(data);
  });

In the snippet, GetDataAsync is expected to do some good work asynchronously on a separate thread while the lambda bound to ContinueWith takes over where, and when, it finishes.

Projecting this pattern to the complexity of a real-world piece of business logic, one runs across a number of issues like callback nesting, exception propagation problems, synchronization context and, last but not least, difficult debugging.

Why Async Code?

The whole point of asynchronous code is the dream of achieving a bit of parallelism or, at the very minimum, save valuable and scarce resources like threads. Overall, there are three types of operations for which optimized/parallel execution would be great:

1. Time-scheduled operations like “do this at time T”
2. I/O-bound operations like a database query or file read/write
3. CPU-bound operations like heavy algorithms that require computing power

Moreover, in a business perspective often the business operations is composed of a combination of multiple of such tasks. In software there’s always a thread of execution moving thing ahead till completion. If not done asynchronously, then this main thread would be waiting indefinitely no matter the amount of power you have on the physical machine.  

For desktop applications, the main problem is keeping the user interface responsive not to annoy the user. For web applications, the main issue is keeping the site responsive enough to serve all other users that concurrently act on it. That’s why critical operations must run asynchronously which ultimately means being offloaded to some other executor. When the main flow spans in multiple threads of execution you face the problem of reconnecting the various pieces together at some later point in time.

Like it or not, concatenating the results of independently running threads is the only way in which, algorithmically speaking, asynchronous code could work in programming: the flow of code at some point stops, awaits for some potentially long running task to finish.

Alternatives in an Ideal World

With the ultimate goal of simplifying the life of developers, the C# sat down looking for ways to make async code easier to write. They faced two options:

1. Rewrite (or at least, significantly restructure and expand) the runtime engine of C# generated code
2. Keep up with the continuation-based model but add a layer of syntactic sugar in the compiler so that the developer’s programming model could be largely simplified while producing the same – but well-written by design – code

Therefore, the async/await feature is backed by compiler support that rewrites asynchronous code into continuation-based logic automatically and correctly.

In an ideal world with no backward compatibility constraints and a runtime designed from scratch, a plausible alternative would be first-class coroutines directly supported by the runtime. In this model, a function could suspend and resume execution natively. Another approach would be lightweight threads (fibers) managed by the runtime. Instead of async methods returning tasks, the runtime could schedule millions of such threads that block without blocking the OS thread. Interestingly, younger than C#, both Go and Kotlin provides coroutines natively. Erlang, instead, supports fibers. From the developer’s standpoint, the code looks sequential while it actually runs in different times and within stop-and-go procedures.

The C# team went for syntactic sugar because that option would require no changes to the CLR, build on top of existing Task class, provide easy migration from existing code and offered a familiar sequential syntax. In other words, async/await was designed to work within the constraints of the existing .NET runtime, not as a theoretically perfect model.

Under the Hood

At a high level, execution of async code can be represented with a state-machine. Before async/await, you were responsible for simulating that through lower-level APIs. With async/await, the compiler does that for you, safely and effectively.  In other programming languages, coroutines are hard-coded like embedded, out-of-the-box state-machines.

Why Async Code?

The whole point of asynchronous code is the dream of achieving a bit of parallelism or, at the very minimum, save valuable and scarce resources like threads. Overall, there are three types of operations for which optimized/parallel execution would be great:

1. Time-scheduled operations like “do this at time T”
2. I/O-bound operations like a database query or file read/write
3. CPU-bound operations like heavy algorithms that require computing power

Moreover, in a business perspective often the business operations is composed of a combination of multiple of such tasks. In software there’s always a thread of execution moving thing ahead till completion. If not done asynchronously, then this main thread would be waiting indefinitely no matter the amount of power you have on the physical machine.  

For desktop applications, the main problem is keeping the user interface responsive not to annoy the user. For web applications, the main issue is keeping the site responsive enough to serve all other users that concurrently act on it. That’s why critical operations must run asynchronously which ultimately means being offloaded to some other executor. When the main flow spans in multiple threads of execution you face the problem of reconnecting the various pieces together at some later point in time.

Like it or not, concatenating the results of independently running threads is the only way in which, algorithmically speaking, asynchronous code could work in programming: the flow of code at some point stops, awaits for some potentially long running task to finish.

Alternatives in an Ideal World

With the ultimate goal of simplifying the life of developers, the C# sat down looking for ways to make async code easier to write. They faced two options:

1. Rewrite (or at least, significantly restructure and expand) the runtime engine of C# generated code
2. Keep up with the continuation-based model but add a layer of syntactic sugar in the compiler so that the developer’s programming model could be largely simplified while producing the same – but well-written by design – code

Therefore, the async/await feature is backed by compiler support that rewrites asynchronous code into continuation-based logic automatically and correctly.

In an ideal world with no backward compatibility constraints and a runtime designed from scratch, a plausible alternative would be first-class coroutines directly supported by the runtime. In this model, a function could suspend and resume execution natively. Conceptually, it would be something like this:

Another approach would be lightweight threads (fibers) managed by the runtime. Instead of async methods returning tasks, the runtime could schedule millions of such threads that block without blocking the OS thread.

Interestingly, younger than C#, both Go and Kotlin provides coroutines natively. Erlang, instead, supports fibers.

From the developer’s standpoint, the code looks sequential while it actually runs in different times and within stop-and-go procedures.

The C# team went for syntactic sugar because that option would require no changes to the CLR, build on top of existing Task class, provide easy migration from existing code and offered a familiar sequential syntax.

In other words, async/await was designed to work within the constraints of the existing .NET runtime, not as a theoretically perfect model.

Under the Hood

At a high level, execution of async code can be represented with a state-machine. Before async/await, you were responsible for simulating that through lower-level APIs. With async/await, the compiler does that for you, safely and effectively.  In other programming languages, coroutines are hard-coded like embedded, out-of-the-box state-machines.

Now, consider such a simple method:

public async Task<int> GetValueAsync()
{
    await Task.Delay(1000);
    return 42;
}

The compiler transforms the method into:

1. A stub method that initializes a state machine.
2. A generated state machine type implementing IAsyncStateMachine.
3. A MoveNext method containing the rewritten body.

The stub roughly becomes:

public Task<int> GetValueAsync()
{
    var stateMachine = new <GetValueAsync>d__0();
    stateMachine.builder = AsyncTaskMethodBuilder<int>.Create();
    stateMachine.state = -1;
    stateMachine.builder.Start(ref stateMachine);
    return stateMachine.builder.Task;
}

The original method signature remains unchanged except that async disappears, since it is only a compile-time marker. The process works as follows:

1. The method starts executing synchronously.
2. When it reaches an await, the compiler-generated code checks whether the awaited operation is already complete.
3. If not complete:
   • The current state is stored.
   • A continuation is registered.
   • The method returns immediately to the caller.
4. When the awaited operation completes, the continuation calls MoveNext() again, resuming execution from the saved state.

Note that newer versions of C# have brought a bunch of optimizations for a better use of memory.

• The state machine is initially created as a struct.
• If the method completes synchronously, it may remain on the stack.
• Only when suspension occurs does it typically move to the heap.

This optimization is one reason why asynchronous methods can be very efficient when they complete synchronously.

Where’s the Trick?

All good except that all this explanation doesn’t say who does wait for the task being offloaded to terminate. The details of this depend on the operating system but Windows and Linux (to name the most popular) do nearly the same.

Time-based operations rely on built-in scheduling services internal to the operating system, at no cost for the application. The executing thread is freed and another one will be picked up to resume execution.

I/O-bound operations rely on external services like disk I/O OS services, database server or remote cloud API for the actual work and on OS TCP services for transport and connection.

CPU-bound operations just use another twin thread in the same process of yours. In a web perspective, it means that it just switches to another pool thread without significant benefits for overall app responsiveness—still one thread busy for long. In a desktop perspective, you have another thread doing work in parallel so for the specific nature of the app this is anyway good.

If you have long-running, CPU-intensive tasks, the best is scheduling a background job or move it to a different remote API.

Summary

The C# compiler implements async/await by rewriting the method into a generated state machine that manages execution across await points, using continuations and task builders to resume execution without blocking threads. This transformation allows asynchronous code to appear sequential while actually executing in a non-blocking, event-driven manner.