Skip to main content

Using TestDriven.NET and NCover to benchmark and monitor unit test code coverage

I wrote this blog post about a year ago - have finally got around to putting it online. It uses a "genericised" system called UpdateManager to demonstrate how to set up code coverage analysis for .NET. Hopefully it's not too dated just yet...

What is unit test code coverage benchmarking/analysis?

Unit test code coverage analysis essentially tells us "how much of my codebase is covered (or "exercised") by my unit test suite?"
When we are writing unit tests, it is necessary to visually scan the codebase and get a feel for what degree of coverage we have and what parts of the codebase are important/relevant for unit test.
Setting up a benchmark such as "a standard .NET codebase must have at least 70% unit test coverage" can be useful, as although this is not a fail-safe way to make sure that we are testing the right things, we can at least be certain that if our metrics indicate we meet the benchmark, most of the important stuff will be tested.
There are several tools available that will provide this sort of metric on a .NET codebase - including NDepend, NCover, etc.

TestDriven.NET vs ReSharper

TestDriven.NET is a tool that is primary intended to support unit test development, but it has a lot of overlap with ReSharper.
If you're already using ReSharper currently (as many .NET development teams do) - and given the fact that ReSharper generally has superior functionality where there is functionality overlap with TestDriven.NET - there is not a lot of value for us in having an additional tool for this purpose.
TestDriven.NET however supports one function that ReShaper lacks - which is unit test code coverage analysis - TestDriven.NET has a built-in instance of NCover, which can be accessed like this:
[Getting to NCover from TestDriven.NET]
When NCover is run across a unit test library, it will run all your tests and provide a code coverage report (percentage coverage of codebase by unit tests), that looks something like this (see the Output window of VS2010):

------ Test started: Assembly: UpdateManagerUnitTests.dll ------ 

2014-06-19 16:18:29,366 [TestRunnerThread] ERROR Logger XxxMetadata.LoadFile - ERROR:
System.ArgumentException: Empty path name is not legal.
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at UpdateManager.Logic.XxxMetadata.LoadFile() in E:\UpdateManager\Logic\XxxMetadata.cs:line 56
2014-06-19 16:18:29,423 [TestRunnerThread] ERROR Logger XxxMetadata.XxxMetadata - LoadFile FAILED


19 passed, 0 failed, 0 skipped, took 10.13 seconds (NUnit 2.6.1).
Following that NCover will open an interactive explorer app/window called “NCoverExplorer” – like this:

[NCoverExplorer window]
Using NCoverExplorer, you can browse the codebase (by namespace) under test that has been analysed by NCover and establish the percentage that each assembly, namespace, class and event method/property is covered by unit test.

[NCoverExplorer window - browse to assembly]

[NCoverExplorer window - browse to namespace, etc]


Setting and adhering to a code coverage benchmark (and tooling up accordingly) is a step along the way to establishing a mature, modern software development practice.
Adopting unit testing as a practice by itself is extremely important. However, code coverage benchmarking and analysis provides the on-going measurement and reporting that is required to be able to sustain the practice.
TestDriven.NET and NCover provide a way to get started with adopting and learning this practice relatively cheaply. TestDriven.NET is free for personal use.


When TestDriven.NET is installed over the top of ReSharper, it can sometimes "replace" some of ReSharper's very nice unit testing automation functions (such as those neat little green dots on the left side of the text editor that will launch a test for you and indicate the most recent result).

[ReSharper - neat little green dots for running tests]
Never fear - in order to get ReSharper's functionality back, just re-install ReSharper back over the top of TestDriven.NET. It's a bit of a "sledge hammer" approach, but you only need to do it once, and it works.




Popular posts from this blog

HOW-TO: Add/edit a field in Team Foundation Server 2012 using Visual Studio 2012

It's been a while since I made a purely technical post...

So, today I wanted to make a change to a Microsoft Team Foundation Server 2012 (TFS2012) instance that I am working with to reflect "Actual" time spent on a task - mainly for reporting purposes, and because I have found in the past that making this minor process adjustment yields a relatively useful metric over the long-term.

I am using the Microsoft Scrum 2.1 Process Template ( for a project that I am working with. So that I don't forget how to do this (again!) I will blog-post the procedure I've used to add this field to the template as a screen-shot-based tutorial, as follows...
Before Assuming you are familiar with the Scrum Process Template (2.1-ish) - open a task and take a look at the "Details" section, as follows:

 This is where I want my "Actual" field to show up.
Get the Power Tools Download and install the latest v…

HOW-TO: Apply a “baseless merge” in Team Foundation Server 2010 (and 2012)

Another purely technical post on TFS...
The scenario We wish to migrate code between branches that do not have a branch/merge relationship, in order to expedite urgent changes being made by a project team, without disrupting on-going BAU development work. Sample branch hierachy/strategy Imagine the following branching strategy in TFS (visible by connecting to TFS via Visual Studio 2010 or 2012):

Essentially you have a "DEV" branch, which has a "QA" branch, which in turn has a "PROD" branch. DEV is the branch that you would be using for BAU development. As a piece of development matures, you move it into QA, where it is tested by your internal QA team. There may be further changes made in DEV that are moved into the QA branch as the QA team pick up issues. Once the QA team are happy with a packaged of changes, they will move them into PROD, which is essentially the hand-over to the customer. The PROD branch represents the software that the customer has.


Eclipse/Android error: "Multiple dex files define [...]"

Wow, I am really going nuts blogging this-evening - 2nd post in less than an hour. 

Anyway this is a particularly nasty error that I keep running into with Eclipse/Android when starting the emulator after I have not run it for a little while. Since I run the risk of permanently forgetting the solution to the problem every time I walk away from my Android project (and thus having to spend a painful hour-or-so digging up the procedure again), I will blog it here, for my benefit, and for the benefit of anyone who may also suffer the same problem.

The gist is that when you start the emulator in debug mode (that is, you hit the button in the following image), you get the following error message come out on the console and a nasty popup telling you nothing more than there is an error with your program and you need to fix it:

[2012-04-06 23:20:57 - Dex Loader] Unable to execute dex: Multiple dex files define Lcom/google/gson/ExclusionStrategy;
[2012-04-06 23:20:57 - SimpleList] Conversion to Dal…