How to Test Roslyn Source Generators

When developing a Roslyn Source Generator, it's important to verify that it works as intended. There are several approaches to testing: you can compare the generated code output (for example, with snapshot testing), or you can go further and ensure that the generated code behaves correctly when executed. In this post, I'll show example for both ways. Note that I strongly recommend to use the second approach, as it ensures that the generated code works as expected, not just that it compiles.

#Testing Generated Code

Let's create the test project for the source generator. The test project will use the Microsoft.CodeAnalysis.CSharp.SourceGenerators.Testing package, which provides a framework for testing source generators.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <RestoreAdditionalProjectSources>https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json</RestoreAdditionalProjectSources>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp.SourceGenerators.Testing" Version="1.1.3-beta1.24423.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="3.1.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="xunit.v3" Version="2.0.3" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\RoslynSourceGeneratorSample\RoslynSourceGeneratorSample.csproj" />
  </ItemGroup>

</Project>

A way to test the output of a source generator is to use the CSharpSourceGeneratorTest class:

[Fact]
public async Task Test()
{
    var context = new CSharpSourceGeneratorTest<SampleSourceGenerator, DefaultVerifier>();
    context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
    context.TestCode = "class Dummy { }";

    // List of expected generated sources
    context.TestState.GeneratedSources.Add((typeof(SampleSourceGenerator), "Sample.g.cs", """
        internal static class Sample
        {
            public const string AssemblyName = "TestProject";
        }
        """));

    await context.RunAsync(TestContext.Current.CancellationToken);
}

This is very similar to testing a Roslyn analyzer. You can find more details about the CSharpAnalyzerTest in my previous post: How to test a Roslyn analyzer

However, this approach only tests that the source generator generates the expected code. It does not test that the generated code actually works. Also, you are testing an implementation detail. The way the code is generated may change, but the functionality of the code should remain the same.

#Testing Generated Code Execution

A better way to test the behavior is to compile the generated code and execute it. The CSharpSourceGeneratorTest class doesn't support this out of the box, but using reflection, you can easily extend it to do so.

[Fact]
public async Task Test2()
{
    var test = new CSharpSourceGeneratorTest<SampleSourceGenerator, DefaultVerifier>();
    test.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
    test.TestCode = "";
    test.TestState.GeneratedSources.Add((typeof(SampleSourceGenerator), "Sample.g.cs", """
        internal static class Sample
        {
            public const string AssemblyName = "TestProject";
        }
        """));

    await Run(test, (alc, assembly) =>
    {
        // Here we can use the assembly to test the generated code
        var value = assembly.GetType("Sample").GetField("AssemblyName")?.GetValue(null);
        Assert.Equal("TestProject", value);
    });
}

// Code to compile the assembly and load it into an isolated AssemblyLoadContext
// Using an ALC allows us to unload the assembly after the test, which is useful for testing multiple source generators in the same test run.
// Also, each ALC is isolated, so we can load different dependencies for each test if needed.
private static async Task Run<TAnalyzer>(CSharpSourceGeneratorTest<TAnalyzer, DefaultVerifier> test, Action<AssemblyLoadContext, Assembly> action)
    where TAnalyzer : new()
{
    using var assemblyStream = await CompileAssembly(test);
    var alc = new AssemblyLoadContext("dummy", isCollectible: true);
    try
    {
        // Note that you may want to load dependencies of the assembly here, if needed using
        // alc.LoadFromAssemblyPath or alc.Resolving += (context, name) => { };
        var assembly = alc.LoadFromStream(assemblyStream);
        action(alc, assembly);
    }
    finally
    {
        alc.Unload();
    }
}

// Note: You can create the compilation manually if you prefer. see the cookbook for more info
// https://github.com/dotnet/roslyn/blob/9eec48b9253e5c65e22dbe4a41cc30191ee6c974/docs/features/source-generators.cookbook.md#unit-testing-of-generators
private static async Task<MemoryStream> CompileAssembly<TAnalyzer>(CSharpSourceGeneratorTest<TAnalyzer, DefaultVerifier> test) where TAnalyzer : new()
{
    var primaryProject = new EvaluatedProjectState(test.TestState, test.ReferenceAssemblies);
    var additionalProjects = test.TestState.AdditionalProjects.Values
        .Select(additionalProject => new EvaluatedProjectState(additionalProject, test.ReferenceAssemblies))
        .ToImmutableArray();

    // Call the method CreateProjectImplAsync in the CSharpSourceGeneratorTest class:
    // protected virtual async Task<Project> CreateProjectImplAsync(EvaluatedProjectState primaryProject, ImmutableArray<EvaluatedProjectState> additionalProjects, CancellationToken cancellationToken)
    var project = await (Task<Project>)test.GetType().GetMethod("CreateProjectImplAsync", BindingFlags.Instance | BindingFlags.NonPublic)
            .Invoke(test, [primaryProject, additionalProjects, TestContext.Current.CancellationToken]);
    var compilation = await project.GetCompilationAsync();

    var generator = ((IIncrementalGenerator)new TAnalyzer()).AsSourceGenerator();
    GeneratorDriver driver = CSharpGeneratorDriver.Create(generator);
    driver = driver.RunGeneratorsAndUpdateCompilation(compilation, out var outputCompilation, out var diagnostics);

    var outputStream = new MemoryStream();
    var result = outputCompilation.Emit(outputStream);
    Assert.True(result.Success, "Compilation failed: " + string.Join(Environment.NewLine, result.Diagnostics.Select(d => d.ToString())));

    outputStream.Seek(0, SeekOrigin.Begin);
    return outputStream;
}

#Testing in a Separate Project

The last way to test a source generator is to create a new project that uses the source generator and test it as a regular project. Sometimes, you may got issues with the IDE support. But it can be definitly easier to test the source generator this way. First, you need to correctly reference the source generator in the test project:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\RoslynSourceGeneratorSample\RoslynSourceGeneratorSample.csproj"
                      PrivateAssets="all"
                      ReferenceOutputAssembly="false"
                      OutputItemType="Analyzer"
                      SetTargetFramework="TargetFramework=netstandard2.0" />

    <PackageReference Include="xunit.runner.visualstudio" Version="3.1.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="xunit.v3" Version="2.0.3" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
  </ItemGroup>

</Project>

Then, you can use the generated code in the test project:

[Fact]
public void Test3()
{
    var value = Sample.AssemblyName;
    Assert.Equal("RoslynSourceGeneratorSample.Tests", value);
}

#Additional Resources

• Code sample

Do you have a question or a suggestion about this post? Contact me!