MSTest v2: Create new asserts

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

MSTest v2 is extensible. In a previous post, we saw how to extend data tests. In this post, we'll see how to create new asserts.

The MSTest framework contains lots of assert methods. They are all located in 3 classes: Assert, StringAssert, CollectionAssert. Sometimes, you want more asserts. In this case you can create your own assert methods.

An assert method is a method that validate a value statifies a specific criteria. If the criteria is not statisfied, an exception is thrown. In MSTest, the exception is of type AssertFailedException, but you can throw any kind of exception.

Let 's create a custom assert to ensure a string is equal to another one. If the 2 strings are differents the message contains a formatted diff that shows blank characters and the first different character:

internal static class MyAssert
{
    public static void StringEquals(string expected, string actual)
    {
        if (expected == actual)
            return;

        throw new AssertFailedException(GetMessage(expected, actual));
    }

    private static string GetMessage(string expected, string actual)
    {
        var expectedFormat = ReplaceInvisibleCharacters(expected);
        var actualFormat = ReplaceInvisibleCharacters(actual);

        // Get the index of the first different character
        var index = expectedFormat.Zip(actualFormat, (c1, c2) => c1 == c2).TakeWhile(b => b).Count();
        var caret = new string(' ', index) + "^";

        return $@"Strings are differents.
Expect: <{expectedFormat}>
Actual: <{actualFormat}>
         {caret}";
    }

    private static string ReplaceInvisibleCharacters(string value)
    {
        return value
            .Replace(' ', '·')
            .Replace('\t', '→')
            .Replace("\r", "\\r")
            .Replace("\n", "\\n");
    }
}

In case the values are differents, the output of the test run look like:

[TestMethod]
public void Test1()
{
    MyAssert.StringEquals("abc def", "abc  def");
}
Message: Strings are differents.
Expect: <abc·def>
Actual: <abc··def>
             ^

Ok, that great. But MSTest v2 provides a way to integrate your assert into the provided asserts. If you look at the intellisense, you may have notice the That property. The property has no member:

So, why the hell is this property? The idea is to provide a property that returns an instance of Assert. This allows you to create extension methods 😃 (of course, the same property exists in StringAssert and CollectionAssert)

So, you can rewrite the assert method as an extension method:

internal static class MyAssert
{
    public static void StringEquals(this Assert assert, string expected, string actual)
    {
        if (expected == actual)
            return;

        throw new AssertFailedException(GetMessage(expected, actual));
    }

    // ...
}

Now, you can use the extension method in the test:

[TestMethod]
public void Test1()
{
    Assert.That.StringEquals("abc def", "abc  def");
}

This syntax is more readable as the line is a valid English sentence.

Conclusion

Writing custom assert method allows to create more useful message, or to create business specific asserts. MSTest v2 provides a nice way to create custom assert by using a extension method. The That property is very smart 😃

Tip: Download Sysinternals tools from the console

Here's a nice tip to download all the Sysinternals tools in 3 command lines. You know all those awesome tools such as Sysmon, Process Explorer, Process Monitor, bginfo, and so on.

net use x: \\live.sysinternals.com
xcopy /s x:\ c:\sysinternals\
net use x: /d

If you just want to use a tool from time to time, you don't need to download them all. Instead, you can open the Run window (Win+R) and enter the full uri of the tool. Of course, you must know the name of the tool! For instance, if you want to run Process Monitor (procmon.exe), you can use the following command:

\\live.sysinternals.com\tools\procmon.exe

Hope this helps!

MSTest v2: Test lifecycle attributes

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

In a test project, some tests may have pre-conditions. You may also do some cleanup. For instance, you need to set a global configuration, or to delete some files after a test ran. This may be more useful for integration tests than for unit tests.

MSTest v2 provides a way to declare methods to be called by the test runner before or after running a test. Here's how to use declare them:

Initialize/Cleanup by assembly

The method decorated by [AssemblyInitialize] is called once before running the tests of the assembly. The method decorated by [AssemblyCleanup] is called after all tests of the assembly are executed. These methods can be located in any class as long as the class is decorated by [TestClass].

[TestClass]
public class Initialize
{
    [AssemblyInitialize]
    public static void AssemblyInitialize(TestContext context)
    {
        Console.WriteLine("AssemblyInitialize");
    }

    [AssemblyCleanup]
    public static void AssemblyCleanup()
    {
        Console.WriteLine("AssemblyCleanup");
    }
}

Initialize/Cleanup by class

The method decorated by [ClassInitialize] is called once before running the tests of the class. In some case, you can write the code in the constructor of the class. The method decorated by [ClassCleanup] is called after all tests of the class are executed.

[TestClass]
public class TestClass1
{
    [ClassInitialize]
    public static void ClassInitialize(TestContext context)
    {
        Console.WriteLine("ClassInitialize");
    }

    [ClassCleanup]
    public static void ClassCleanup()
    {
        Console.WriteLine("ClassCleanup");
    }

    [TestMethod]
    public void Test1()
    {
    }
}

Initialize/Cleanup by test

The method decorated by [TestInitialize] is called before running each test of the class. The method decorated by [TestCleanup] is called after running each test of the class.

[TestClass]
public class TestClass1
{
    [TestInitialize]
    public void TestInitialize()
    {
        Console.WriteLine("TestInitialize");
    }

    [TestCleanup]
    public void TestCleanup()
    {
        Console.WriteLine("TestCleanup");
    }

    [TestMethod]
    public void Test1()
    {
    }
}

Execution log

Here's the execution log. I think this is much more clearer than long sentenses!

AssemblyInitialize (once by assembly)
  ClassInitialize  (once by class)
    TestInitialize (before each test of the class)
      Test1
    TestCleanup    (after each test of the class)
    TestInitialize
      Test2
    TestCleanup
    ...
  ClassCleanup     (once by class)
  ClassInitialize
      ...
  ClassCleanup
AssemblyCleanup    (once by assembly)

Validation made easy with decorators

Before reading this post, you should read the previous post about Aspect Oriented Programming in TypeScript. It will help you understand the TypeScript decorators.

The idea of this post is to write a class and use decorators to set the validation rules on attributes:

class Customer {
    @required
    public firstName: string;
    @required
    public lastName: string;
}

var customer = new Customer();
customer.firstName = 'Gérald';
validate(customer); // 'lastName' is required

The solution is to use decorators and metadata to attach the validation rules to the associate properties. Then, you can query these metadata to run the rules. TypeScript supports the Metadata Reflection API. This API defines a few methods. Here's are the ones we'll use:

namespace Reflect {
    // Set metadata

    // Reflect.defineMetadata("custom:annotation", value, Customer);
    function defineMetadata(metadataKey: any, metadataValue: any, target: Object): void;
    // Reflect.defineMetadata("custom:annotation", value, Customer.prototype, "firstName");
    function defineMetadata(metadataKey: any, metadataValue: any, target: Object, propertyKey: string | symbol): void;

    // Get metadata

    // Reflect.getMetadata("custom:annotation", Customer);
    function getMetadata(metadataKey: any, target: Object): any;
    // Reflect.getMetadata("custom:annotation", Customer.prototype, "firstName");
    function getMetadata(metadataKey: any, target: Object, propertyKey: string | symbol): any;
}

The following code attach a metadata named "validation" to the property and the class. The metadata for the property is a list of ValidationRule. For the class we attach the list of properties to validate.

First, you need to install the metadata polyfill:

npm install reflect-metadata

Then, let's create a function to add the metadata to the property and class:

import "reflect-metadata";

function addValidationRule(target: any, propertyKey: string, rule: IValidationRule) {
    let rules: IValidationRule[] = Reflect.getMetadata("validation", target, propertyKey) || [];
    rules.push(rule);

    let properties: string[] = Reflect.getMetadata("validation", target) || [];
    if (properties.indexOf(propertyKey) < 0) {
        properties.push(propertyKey);
    }

    Reflect.defineMetadata("validation", properties, target);
    Reflect.defineMetadata("validation", rules, target, propertyKey);
}

Let's implement the required validation rule:

interface IValidationRule {
    evaluate(target: any, value: any, key: string): string | null;
}

class RequiredValidationRule implements IValidationRule {
    static instance = new RequiredValidationRule();

    evaluate(target: any, value: any, key: string): string | null {
        if (value) {
            return null;
        }

        return `${key} is required`;
    }
}

function required(target: any, propertyKey: string) {
    addValidationRule(target, propertyKey, RequiredValidationRule.instance);
}

Finally, you can validate an object using the metadata:

function validate(target: any) {
    // Get the list of properties to validate
    const keys = Reflect.getMetadata("validation", target) as string[];
    let errorMessages: string[] = [];
    if (Array.isArray(keys)) {
        for (const key of keys) {
            const rules = Reflect.getMetadata("validation", target, key) as IValidationRule[];
            if (!Array.isArray(rules)) {
                continue;
            }

            for (const rule of rules) {
                const error = rule.evaluate(target, target[key], key);
                if (error) {
                    errorMessages.push(error);
                }
            }
        }
    }

    return errorMessages;
}

function isValid(target: any) {
    const validationResult = validate(target);
    return validationResult.length === 0;
}

Conclusion

In the previous post, I showed how to use decorators to change the default behavior of a method or class. Metadata allows to use decorators for new scenario. If you look at it, it really similar to C#.

MSTest v2: Data tests

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

In the previous post, I showed you the basis of writing a unit test with MSTest v2. There was only one unit test for the MathHelper.Add method. Of course, you want to write more tests to validate the behavior of the method. One way is to copy and paste the method and change the value of the parameters.

[TestClass]
public class MathTests
{
    [TestMethod]
    public void Test_Add1()
    {
        var actual = MathHelper.Add(1, 1);
        var expected = 2;
        Assert.AreEqual(expected, actual);
    }

    [TestMethod]
    public void Test_Add2()
    {
        var actual = MathHelper.Add(21, 4);
        var expected = 25;
        Assert.AreEqual(expected, actual);
    }
}

As a developper, you may avoid copy pasting code everytimes it's possible! This idea is to create a parametrized test. This means the test methods have parameters and uses them to parametrize the test. This way, one test method can be use to run N tests. MSTest v2 provides 3 ways to create parametrized tests.

Using DataRow

The [DataRow] attribute allows to set the values of the parameter of the test. You can set as many [DataRow] attributes as you want. Also you need to replace [TestMethod] by [DataTestMethod]:

[TestClass]
public class MathTests
{
    [DataTestMethod]
    [DataRow(1, 1, 2)]
    [DataRow(12, 30, 42)]
    [DataRow(14, 1, 15)]
    public void Test_Add(int a, int b, int expected)
    {
        var actual = MathHelper.Add(a, b);
        Assert.AreEqual(expected, actual);
    }
}

You can see that there are 3 tests in the test explorer.

Using DynamicData

If your data cannot be set into an attribute parameters (non constant values or complex objects), you can use the [DynamicData] attribute. This attribute allows to get the values of the parameters from a method or a property. The method or the property must return an IEnumerable<object[]>. Each row corresponds to the values of a test.

Here's an example with a method:

[TestClass]
public class MathTests
{
    [DataTestMethod]
    [DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
    public void Test_Add_DynamicData_Method(int a, int b, int expected)
    {
        var actual = MathHelper.Add(a, b);
        Assert.AreEqual(expected, actual);
    }

    public static IEnumerable<object[]> GetData()
    {
        yield return new object[] { 1, 1, 2 };
        yield return new object[] { 12, 30, 42 };
        yield return new object[] { 14, 1, 15 };
    }
}

Here's an example with a property:

[TestClass]
public class MathTests
{
    [DataTestMethod]
    [DynamicData(nameof(Data), DynamicDataSourceType.Property)]
    public void Test_Add_DynamicData_Property(int a, int b, int expected)
    {
        var actual = MathHelper.Add(a, b);
        Assert.AreEqual(expected, actual);
    }

    public static IEnumerable<object[]> Data
    {
        get
        {
            yield return new object[] { 1, 1, 2 };
            yield return new object[] { 12, 30, 42 };
            yield return new object[] { 14, 1, 15 };
        }
    }
}

Custom DataSource

If the 2 previous attributes don't fit your needs, you can create your own attribute. The attribute must implement ITestDataSource. This interface has 2 methods: GetData and GetDisplayName. GetData returns the data rows. GetDisplayName returns the name of the test for a data row. This name is visible in the Test Explorer or in the console.

public class CustomDataSourceAttribute : Attribute, ITestDataSource
{
    public IEnumerable<object[]> GetData(MethodInfo methodInfo)
    {
        yield return new object[] { 1, 1, 2 };
        yield return new object[] { 12, 30, 42 };
        yield return new object[] { 14, 1, 15 };
    }

    public string GetDisplayName(MethodInfo methodInfo, object[] data)
    {
        if (data != null)
            return string.Format(CultureInfo.CurrentCulture, "Custom - {0} ({1})", methodInfo.Name, string.Join(",", data));

        return null;
    }
}

Then, you can use this attribute as any other data attributes:

[DataTestMethod]
[CustomDataSource]
public void Test_Add(int a, int b, int expected)
{
    var actual = MathHelper.Add(a, b);
    Assert.AreEqual(expected, actual);
}

Conclusion

You can easily create a lots of unit tests using parametrized tests. The 2 attributes DataRow and DynamicData should be enough for most of the cases. The data tests are extensibles, so you can create your own attributes to create your own data source. Of course, you can combine the 3 methods for the same test method.