MSTest v2: Exploring asserts

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

In the previous post, we created a simple test and run it. You write a test to assert your code behave as expected. This means, assertions are central to unit testing in any frameworks, and MSTest is no exception. MSTest provides a rich set of assertions. So, today we'll explore the Assert classes.

An assertion method is a method that takes paramters and ensure a condition is met. For instance, AreEqual takes 2 values and check they are equal. If an assertion fails, the method call does not return and an error is reported. The method throws an exception with a formatted message. If a test contains multiple assertions, any that follow the one that failed will not be executed.

[TestMethod]
public void Test()
{
    Assert.AreEqual(10, 5 + 5); // Success
    Assert.AreEqual(10, 1 + 2); // 10 ≠ 3 => Error.
    Assert.AreEqual(20, 1 + 2); // Not executed
}

Now, we can explore the different assert methods of the framework.

Assertion on a single value

The following assert methods validate a value meet a specific condition. I won't explain all of them because the usage is pretty clear.

  • Assert.AreEqual(object expected, object actual): Tests whether the specified values are equal. Different numeric types are treated as unequal even if the logical values are equal. 42L is not equal to 42.
  • Assert.AreNotEqual(object expected, object actual): Tests whether the specified values are unequal. Same as AreEqual for numeric values.
  • Assert.AreSame(object expected, object actual): Tests whether the specified objects both refer to the same object
  • Assert.AreNotSame(object expected, object actual): Tests whether the specified objects refer to different objects
  • Assert.IsTrue(bool condition): Tests whether the specified condition is true
  • Assert.IsFalse(bool condition): Tests whether the specified condition is false
  • Assert.IsNull(object value): Tests whether the specified object is null
  • Assert.IsNotNull(object value): Tests whether the specified object is non-null
  • Assert.IsInstanceOfType(object value, Type type): Tests whether the specified object is an instance of the expected type
  • Assert.IsNotInstanceOfType(object value, Type type): Tests whether the specified object is not an instance of the wrong type
  • StringAssert.Contains(string value, string substring): Tests whether the specified string contains the specified substring
  • StringAssert.StartsWith(string value, string substring): Tests whether the specified string begins with the specified substring
  • StringAssert.Matches(string value, Regex regex): Tests whether the specified string matches a regular expression
  • StringAssert.DoesNotMatch(string value, Regex regex): Tests whether the specified string does not match a regular expression

I'll only provide an example for AreEqual and AreSame because the difference is more subtile.

var expected = "a";
var actual = "a";

Assert.AreEqual(expected, actual);    // success
Assert.AreSame(expected, actual);     // fail

Assert.AreNotEqual(expected, actual); // fail
Assert.AreNotSame(expected, actual);  // success

Assert.AreEqual(42, 42);              // success
Assert.AreEqual(42, 42L);             // fail

Assertion on collections

  • CollectionAssert.AreEqual(ICollection expected, ICollection actual): Tests whether the specified collections have the same elements in the same order and quantity.
  • CollectionAssert.AreNotEqual(ICollection expected, ICollection actual): Tests whether the specified collections does not have the same elements or the elements are in a different order and quantity.
  • CollectionAssert.AreEquivalent(ICollection expected, ICollection actual): Tests whether two collections contain the same elements.
  • CollectionAssert.AreNotEquivalent(ICollection expected, ICollection actual): Tests whether two collections contain different elements.
  • CollectionAssert.AllItemsAreInstancesOfType(ICollection collection, Type expectedType): Tests whether all elements in the specified collection are instances of the expected type
  • CollectionAssert.AllItemsAreNotNull(ICollection collection): Tests whether all items in the specified collection are non-null
  • CollectionAssert.AllItemsAreUnique(ICollection collection): Tests whether all items in the specified collection are unique
  • CollectionAssert.Contains(ICollection collection, object element): Tests whether the specified collection contains the specified element
  • CollectionAssert.DoesNotContain(ICollection collection, object element): Tests whether the specified collection does not contain the specified element
  • CollectionAssert.IsSubsetOf(ICollection subset, ICollection superset): Tests whether one collection is a subset of another collection
  • CollectionAssert.IsNotSubsetOf(ICollection subset, ICollection superset): Tests whether one collection is not a subset of another collection

Here's an example to show the difference between AreEqual and AreEquivalent:

CollectionAssert.AreEqual(new [] { 1, 2 }, new [] { 1, 2 });    // success
CollectionAssert.AreEqual(new [] { 1, 2 }, new [] { 1, 2, 3 }); // fail
CollectionAssert.AreEqual(new [] { 1, 2 }, new [] { 2, 1 });    // fail

CollectionAssert.AreEquivalent(new [] { 1, 2 }, new [] { 1, 2 });    // success
CollectionAssert.AreEquivalent(new [] { 1, 2 }, new [] { 1, 2, 3 }); // fail
CollectionAssert.AreEquivalent(new [] { 1, 2 }, new [] { 2, 1 });    // success

Test an exception is thrown

  • Assert.ThrowsException<T>(Action action): Tests whether the code specified by delegate throws exact given exception of type T (and not of derived type)
  • Assert.ThrowsExceptionAsync<T>(Func<Task> action): Same as ThrowsException with async code
// success
Assert.ThrowsException<ArgumentNullException>(() => new Regex(null));

// fail
Assert.ThrowsException<ArgumentException>(() => new Regex(null));

Other asserts

The following assert methods are special. The first one Fail immediately change the status of the test to "error". This can be useful to write your own assert method. The second one, Inconclusive, indicates that the test could not be completed. In this case, the test is neither a success nor a failure.

  • Assert.Fail(string message)
  • Assert.Inconclusive(string message)

Here's an example of an inconclusive test

[TestMethod]
public void Sample8()
{
    if (DateTime.Now.Hour != 12)
        Assert.Inconclusive("Must be executed at noon");

    // actual tests...
}

In the test explorer, inconclusive tests are grouped into "Skipped Tests":

Conclusion

MSTest v2 comes with more than 30 assert methods. This should helps you writting your tests. In a future blog post, we'll see how to extend the list of assert methods with your own methods.

Import polyfill using dynamic imports in TypeScript

TypeScript 2.4 has introduced Dynamic imports. In a previous, we have seen how to use static imports in TypeScript. Static import allow to use a module. You import all the modules needed at the beginning of the TypeScript file and use them later:

import { ZipCodeValidator } from "./ZipCodeValidator";
const myValidator = new ZipCodeValidator();

Dynamic imports on the other hand are not declared at the beggining of the file, but right before their usage. This is very useful to load a module only when you really need it. For instance, if a feature is not commonly use, you can load the code of that feature only when the user starts using it. This way, when the page loads, the user don't need to download all the code, so it reduces the time to load the page.

if (condition) {
    let zipCodeValidator = await import("./ZipCodeValidator");
    let myValidator = new zipCodeValidator();
}

What is a polyfill?

From wikipedia:

In web development, a polyfill is code that implements a feature on web browsers that do not support the feature. Most often, it refers to a JavaScript library that implements an HTML5 web standard, either an established standard (supported by some browsers) on older browsers, or a proposed standard (not supported by any browsers) on existing browsers. Formally, "a polyfill is a shim for a browser API".

Polyfills allow web developers to use an API regardless of whether it is supported by a browser or not, and usually with minimal overhead. Typically they first check if a browser supports an API, and use it if available, otherwise using their own implementation.

For instance, String.includes is not available in Internet Explorer 11. This is a very convenient function, so I want to be able to use it. The idea is to run the polyfill before the rest of the code, so you can use the function. You can find a polyfill on MDN.

// Test if String.includes is available
if (!String.prototype.includes) {

    // Add the find method to the prototype
    String.prototype.includes = function(search, start) {
        'use strict';
        if (typeof start !== 'number') {
            start = 0;
        }

        if (start + search.length > this.length) {
            return false;
        } else {
            return this.indexOf(search,start) !== -1;
        }
    };
}

Load polyfills dynamically

The previous polyfill is not very long. However, if you need to include ten polyfills or a much longer one, you'll increase the size of the JS, and your web site will be slower. Plus, String.includes is well supported by the current browsers. You can check that information on caniuse:

So, you need this polyfill for just a few users. You don't want to impact the vast majority of your audiance. So, why not loading this polyfill dynamically. This is where dynamic import is useful. Indeed, you can create the polyfill as a module and import it only when needed.

Here's the structure I use:

src
├── polyfills
│   ├── array.from.js
│   ├── array.prototype.find.js
│   ├── string.prototype.includes.js
│   └── ...
└── main.ts

Polyfills are often paste from the internet, so you can use JavaScript or TypeScript depending on the source.

Then, you can import them dynamically if needed in the main file:

function importPolyfills() {
    let promises = [];
    if (!String.prototype.includes) {
        promises.push(import("./polyfills/string.prototype.includes"));
    }

    if (!Array.from) {
        promises.push(import("./polyfills/array.from"));
    }

    if (!Array.prototype.find) {
        promises.push(import("./polyfills/array.prototype.find"));
    }

    // ...

    return Promise.all(promises);
}

importPolyfills().then(() => {
    console.log("meziantou".includes("an"));
});

This way, only the users who use a browser that don't support the feature you want will actually download the polyfill. For them, the loading time will increase a little. But, for the vast majorty of the users, it will reduce the loading time. And when a browser releases the functionality, the polyfill won't be downloaded anymore. So, you can measure how many people download the polyfill file and check if it is actually needed.

MSTest v2: Setup a test project and run tests

This post is part of the serie 'MSTest v2'. Be sure to check out the rest of the blog posts of the serie!

Unit testing is part of the development process. If you are not writing much unit tests now, I would recommend reading this post which explains how testing helps to deliver faster and with more confidence: How we approach testing VSTS to enable continuous delivery . In this serie of posts, we'll see how to write tests using MS Test v2, the Microsoft's test framework.

Create the test project

In Visual Studio, create a new "Unit Test Project" project:

If you want to use the command line in .NET Core, you can use the following command:

dotnet new mstest

The template contains the references to the following NuGet packages:

  • Microsoft.NET.Test.Sdk
  • MSTest.TestAdapter
  • MSTest.TestFramework

The csproj file should contains the following lines:

<ItemGroup>
  <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
  <PackageReference Include="MSTest.TestAdapter" Version="1.2.0" />
  <PackageReference Include="MSTest.TestFramework" Version="1.2.0" />
</ItemGroup>

Create a test

To create a test, you have to write a method with the attribute [TestAttribute] in a class decorated with the [TestClass] attribute. The TestClass attribute may seem superfluous, but we'll see in another post its importance 😃 You can use the methods of the Assert class to validate the behavior of the method under test.

using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass]
public class MathTests
{
    [TestMethod]
    public void Test_Add()
    {
        var actual = MathHelper.Add(1, 1);
        var expected = 2;
        Assert.AreEqual(expected, actual);
    }
}

public class MathHelper
{
    public static int Add(int a, int b) => a + b;
}

Note: The first argument of the Assert.AreEqual method is the expected value. If you swap the 2 arguments the error message may be weird.

You'll find lots of assert methods in the classes Assert, CollectionAssert and StringAssert. We'll cover then in one of the next posts. It's not mandatory to use the AssertXXX classes. In fact a test fails when an exception is raised. Assert throws exception with a formatted message containing the expected and actual values. By using them you will get readable error message in the test result.

Run the tests from Visual Studio

Now, you can run the test directly in Visual Studio. Open the Test Explorer window (Ctrl+E, T)

Since Visual Studio 2017 Update 5, the tests of the solution are discovered in real time. But if the tests are not visible, building the solution should make them visible. Once the test are visible, click the "Run All" button:

After a short time, the result of the test run is visible:

By default the tests are grouped by output result, so you can quickly find failed tests.

Run the tests from the command line

If you are using .NET Core, you can run tests using the following command:

dotnet test

The full documentation of dotnet test is available in the documentation.

Conclusion

You now have the basis to create and run unit tests written with MSTest v2. In the next posts, we'll explore the functionalities of MSTest v2 and Visual Studio. Stay tuned!

How to migrate from JavaScript to TypeScript?

After the previous posts about TypeScript, I'm sure you'd like to migrate your application from JavaScript to TypeScript. If you haven't read them yet, take a few minutes:

The migration from JavaScript to TypeScript is not a very complicated process. But there are some steps to follow to achieve it correctly without spending too much time.

Learn the TypeScript language

Of course, you have to start learning the language. If you are already a JavaScript developer, learning TypeScript is easy.

Here's a few good links:

You'll also find useful resources on this blog 😃

Install TypeScript

If you are using Visual Studio, the TypeScript compiler is already installed. In other cases, you must install it.

npm install typescript
.\node_modules\.bin\tsc --version

or you can install it globally

npm install -g typescript
tsc --version

Add the TypeScript configuration file to your project

The configuration file indicates to the TypeScript compiler the options to use to compile the project. This is also the root of the TypeScript project.

Your solution should look like:

Root
├── src
│   ├── file1.js
│   └── file2.js
├── built
└── tsconfig.json

The tsconfig.json at the root of the project contains:

{
  "compilerOptions": {
    "target": "es5",
    "allowJs": true,
    "outDir": "./built"
  },
  "include": [
      "./src/**/*"
  ]
}

The file indicates the TypeScript compiler to accept JavaScript files, so you don't need to change all your code at once. With this option, you can convert your files one by one. The outDir indicates where TypeScript outputs the files after the compilation. The include options indicates where are located the source files.

By default, the compiler won't analyse the error in the JS files. You can start getting some of the bemefits of TypeScript by indicates the compiler to check your JS files using "checkJs": true:

{
  "compilerOptions": {
    "target": "es5",
    "allowJs": true,
    "checkJs": true,
    "outDir": "./built"
  },
  "include": [
      "./src/**/*"
  ]
}

The compiler will use the information it can gather from your files to indicate errors (documentation). For instance, it can use JSDoc to find types or resolve require("...") statements. So, You'll get some of the TypeScript advantages immediately. You can also set some compiler options such as noImplicitReturns: true, noFallthroughCasesInSwitch: false, allowUnreachableCode: false or allowUnusedLabels: false.

/** @type {string} */
var author;

author = "meziantou"; // OK
author = false;       // Error: boolean is not assignable to string

Convert files to TypeScript and fix common errors

It's time to start using TypeScript. The only change required is to change the extension a the files from .js to .ts. You don't need to migrate all files at once. You can migrate files one by one. If you have a large code base, you'll probably get hundreds of errors. So, converting files one by one will allow to handle them more easily.

Some of valid JavaScript will stop working in TypeScript. Mainly because of the type checker. Here's a few errors you may encounter.

Sequentially Added Properties

The following code is very common in JavaScript:

var author = {};
author.nickname = "Meziantou";
author.fullName = "Gérald Barré";

In TypeScript the type of author is an empty object. So nickname and fullname doesn't exist and are not assignable. The solution is to move the declarations into the object creation:

var author = {
    nickname = "Meziantou",
    fullName = "Gérald Barré"
};

The other solution is to create a type for the author variable, so the compiler knows the list of properties available.

interface Author { nickname: string; fullName: string }

var author = {} as Author;
author.nickname = "Meziantou";
author.fullName = "Gérald Barré";

The latest option, which is not the best, is to indicate the compiler to not check the usages of the author variable using the type any:

var author: any = {};
author.nickname = "Meziantou";
author.fullName = "Gérald Barré";

Using a well-know library

TypeScript must know the types of every objects to compile a file. So, if you are using a library such as JQuery, loadash, etc., you have to tell TypeScript what the library contains. Fortunately, the TypeScript definitions of thousands of lib are available via npm. For instance, if you use jquery, you can install the definition using

npm install @types/jquery

You'll find the complete list of supported libraries on npm: https://www.npmjs.com/~types

Using others libraries

If the types are not available, you have 2 options:

  • Create the definition of the methods/objects
  • Create a dummy declaration

For instance, you use a lib that exposes a function add(a, b) and an object _, you can create the following dummy declaration, so the compiler is happy. The any type indicates to the compiler to not check the value.

declare function add(...args: any[]): any;
declare var _: any;

add(1, 2);
_.filter(characters, { 'age': 36 });

With these definitions, anything after _ will be valid. While this is not the philosophy of TypeScript, it's better to have 80% of the code checked instead of 0%. So, it's a good start. Later, you can write a better definition such as:

declare function add(a: number, b: number): number;
declare var _: Underscore;

interface Underscore {
    filter(obj: any[], filter: { [name: string]: any });
}

External modules (require, define)

If you are using modules with commonjs, requirejs or amd, you can use a dummy declaration:

declare function require(path: string): any;
declare function define(...args: any[]): any;

However, you should use the TypeScript syntax. This is more convenient and you'll get the typing information. First, change the module option in the tsconfig.json file:

{
  "compilerOptions": {
    "module": "commonjs" // Set the kind of module you are using
  }
}

Then change your import statements:

// JavaScript
var math = require("./math");

// TypeScript (one of the following)
import math = require("./math");
math.add(1, 2);

import { add } from "./math";
add(1, 2);

You can also replace the exports with the new syntax

// JavaScript
module.exports.add = function(a, b) {
    return a + b;
}

// TypeScript
export function add(a, b) {
    return a + b;
}

Use new syntaxes

You know have a valid TypeScript project. You can start using the new TypeScript language features. This should help you writing more readable code. For instance, you can:

  • Replace immediately-invoked function expressions with namespace/module
  • Use classes when possible
  • Replace var with let or const
  • Replace magic numbers/strings with enumeration
  • Replace require() with import (check the previous section)
  • Use async await to simplify the usage of asynchronous code
  • Replace property names with nameof equivalent

Some tools such as Resharper allows to replace common JS syntax with a TypeScript equivalent: https://blog.jetbrains.com/dotnet/2015/02/05/ways-and-advantages-of-migrating-javascript-code-to-typescript/

Enable compiler checks to find more errors

Finally, you should change the compiler options to enable more and more checks. This will help you find more errors at compile time. Some of them such as strictNullChecks will generates lots of errors (maybe thousands). So, you should enable these option one by one and fix the new errors.

{
    "compilerOptions": {
        "allowUnreachableCode": false,
        "allowUnusedLabels": false,
        "alwaysStrict": true,
        "noImplicitAny": true,
        "noImplicitReturns": true,
        "noImplicitThis": true,
        "noFallthroughCasesInSwitch": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "suppressExcessPropertyErrors": false,
        "suppressImplicitAnyIndexErrors": false,
        "strictFunctionTypes": true,
        "strictNullChecks": true,
        "strictPropertyInitialization": true,
        "checkJs": true // If you are still using some js files
    }
}

You can read more about all these options in a previous post: Detect common JavaScript errors with TypeScript.

Live unit testing in Visual Studio 2017 Enterprise

Unit tests allow to validate the behavior of your code. When you write code, you want to get feedback from your tests as soon as possible. It is where the new feature of Visual Studio 2017 takes place. The live unit testing feature runs test in background as soon as you save a file and display the result directly in the code editor. Of course you don't want to run thousands of tests in background every time you change a single line of code, but only the test impacted by your changes. Live unit testing knows which part of the code each test cover, so it can run only the needed tests. Thus, you get a feedback very quickly. Let's see how to use live unit testing.

You can enable live unit testing automatically when you open the solution in the options window:

Let's use a simple method to test:

public static string Nullify(string str, bool trim)
{
    if (str == null)
        return null;

    if (trim)
    {
        str = str.Trim();
    }

    if (string.IsNullOrEmpty(str))
        return null;

    return str;
}

After enabling live unit testing, Visual Studio adds a blue dash in front of each line of code of the method. This means the statement is not tested.

Let's add a test and save the file. The info will change to green check for the covered statements. The blue dash stays for uncovered lines.

Now, let's add a test that fails. A red check is added in front of the lines of code. This indicator override other indicators. By clicking on a red cross, you can see the details of each test that covers the selected line.

Now, you can easily debug the failing test by clicking on the red cross:

And you can see the details by hovering a specific test:

Conclusion

Live unit testing is very nice! You can quickly test your code and see as soon as you save the file where which tests success or fail. You can also know which part of your code is covered by tests and which tests cover them. The only drawback is that this functionality is only available in the enterprise edition of Visual Studio 2017 😦