MSTest v2: Execute tests in parallel

  • .NET

This post is part of the series 'MSTest v2'. Be sure to check out the rest of the blog posts of the series!

By default the MSTest runner executes the tests of an assembly sequentially. If your tests are well isolated you can run them in parallel to reduce the execution time. Since version 1.3, MSTest v2 provides this functionality, but you have to opt-in.

To opt-in for parallel execution, you just have to set the [Parallelize] attribute in the assembly:

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

The attribute has 2 properties: Workers et Scope.

  • Workers: The number of threads to run the tests. Set it to 0 to use the number of core of your computer.
  • Scope: Determine if the runner must parallelize tests at method or class level. MethodLevel will run all tests in parallel. ClassLevel will run all test classes in parallel, but tests in a class are run sequentially. You should use ClassLevel if the tests within classes have interdependencies.

If you have multiple test assemblies you want to parellize you can create a file named .runsettings at the root of the solution:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <MSTest>
    <Parallelize>
      <Workers>8</Workers>
      <Scope>MethodLevel</Scope>
    </Parallelize>
  </MSTest>
</RunSettings>

If you want to get the maximum performance, you should use Workers = 0 and Scope = ExecutionScope.MethodLevel. Note that the scope is MethodLevel. A method can contains multiple tests if you use data test. Each row of a data test is executed sequentially.

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [DataTestMethod]
        [DataRow(1)]
        [DataRow(2)]
        [DataRow(3)]
        public void Test3(int i) => Thread.Sleep(5000);
    }
}

The 3 test methods are executed in parallel, but the 3 tests of Test3 are run sequentially. So, the total execution time will be 15 seconds.

Enabling parallelization is opt-in. Once it's enabled, you can opt-out for a specific test or class using [DoNotParallelize].

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        [DoNotParallelize]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [TestMethod]
        public void Test3() => Thread.Sleep(5000);
    }
}

The runner will execute Test2 and Test3 in parallel. Then, it will execute Test1.

Conclusion

In most cases, you can reduce the time required to run your tests by adding one attribute in your project 😃

Follow me:
Enjoy this blog?Buy Me A CoffeeDonate with PayPal

Comments

Terry Watts -

mmm I can get the tests to run in parallel, but I need one to run on its own. However VS2019 seems to ignore my efforts:

using Microsoft.VisualStudio.TestTools.UnitTesting;

// run tests in parallel
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace FraxesUnitTests
{
...
   [DoNotParallelize]
   [ExpectedException(typeof(ArgumentException))]
   public void GivenCalculate_WhenNoTranslationData_ThenExpectException()
   {
   ..
   }

It fails because it is running in parallel.

Any ideas?

Gérald Barré -

@Terry, Your code looks ok. You should ask the question on Stack Overflow or open an issue on GitHub.

Leave a reply