MSTest v2: Execute tests in parallel

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

By default the MSTest runner executes the tests of an assembly sequentially. If your tests are well isolated you can run them in parallel to reduce the execution time. Since version 1.3, MSTest v2 provides this functionality, but you have to opt-in.

To opt-in for parallel execution, you just have to set the [Parallelize] attribute in the assembly:

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

The attribute has 2 properties: Workers et Scope.

  • Workers: The number of threads to run the tests. Set it to 0 to use the number of core of your computer.
  • Scope: Determine if the runner must parallelize tests at method or class level. MethodLevel will run all tests in parallel. ClassLevel will run all test classes in parallel, but tests in a class are run sequentially. You should use ClassLevel if the tests within classes have interdependencies.

If you have multiple test assemblies you want to parellize you can create a file named .runsettings at the root of the solution:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <MSTest>
    <Parallelize>
      <Workers>8</Workers>
      <Scope>MethodLevel</Scope>
    </Parallelize>
  </MSTest>
</RunSettings>

If you want to get the maximum performance, you should use Workers = 0 and Scope = ExecutionScope.MethodLevel. Note that the scope is MethodLevel. A method can contains multiple tests if you use data test. Each row of a data test is executed sequentially.

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [DataTestMethod]
        [DataRow(1)]
        [DataRow(2)]
        [DataRow(3)]
        public void Test3(int i) => Thread.Sleep(5000);
    }
}

The 3 test methods are executed in parallel, but the 3 tests of Test3 are run sequentially. So, the total execution time will be 15 seconds.

Enabling parallelization is opt-in. Once it's enabled, you can opt-out for a specific test or class using [DoNotParallelize].

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        [DoNotParallelize]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [TestMethod]
        public void Test3() => Thread.Sleep(5000);
    }
}

The runner will execute Test2 and Test3 in parallel. Then, it will execute Test1.

Conclusion

In most cases, you can reduce the time required to run your tests by adding one attribute in your project 😃

Leave a reply