Mutation Testing in C#: Breaking Code to Make It Better

When Tests Pass, But the Code Is Still "Broken"

Imagine this: you've written unit tests for a C# project, dotnet test gives a green light, code coverage is 85%. The team is happy, CI/CD runs perfectly. But a month later, a bug appears in production that, it seems, the unit tests should have caught. Sound familiar?

The problem is that code coverage is a deceptive metric. It shows which lines of code were touched, but it doesn't check the quality of the checks themselves. You can have 100% coverage but not a single meaningful assert. This is where mutation testing in C# comes into play — a technique that "breaks" your code to check how well your tests can catch errors.

In this article, we'll break down what mutation testing is, how to implement it in .NET projects using Stryker.NET, and why it's a must-have for any serious C# developer.

What is Mutation Testing?

Mutation testing is a method for evaluating test quality based on introducing small, deliberate changes (mutations) into the program's source code. Each such modified version of the code is called a mutant.

The logic is simple:

\n
• Your production code is taken.
\n
• Mutants are automatically generated — for example, the + operator is replaced with -, the condition if (x > 5) is changed to if (x >= 5), and true is changed to false.
\n
• Your unit tests are run against each mutant.
\n
• If a test fails — the mutant is "killed". Good test.
\n
• If a test passes — the mutant "survived". Bad sign: your test didn't notice the logic change.
\n

The goal is to achieve a high percentage of killed mutants (typically 80%+). This is real assurance that your tests verify behavior, not just "run through" the code.

Why is Code Coverage Insufficient?

Let's look at a simple C# example. Suppose we have a method that checks if a number is even:

public class MathUtils\n{\n    public bool IsEven(int number)\n    {\n        return number % 2 == 0;\n    }\n}

And a "perfect" test with 100% coverage:

[Test]\npublic void IsEven_ShouldReturnTrueForEvenNumber()\n{\n    var utils = new MathUtils();\n    var result = utils.IsEven(4);\n    Assert.IsTrue(result);\n}

The test is green, coverage is 100%. But what if someone accidentally (or maliciously) changes the method to return number % 2 == 1;? Will our single test fail? No! It will check the number 4 (4 % 2 == 0, but now the method will return false), the assert will fail — the mutant will be killed. All good? Wait.

What if the code is changed to return number % 2 != 1;? Logically it's the same, but mutation testing will create a mutant: return number % 2 != 0;. Our test with the number 4 will still pass (4 % 2 != 0 — false, the method will return false, but the assert expects true — the test will fail). That's fine here. But imagine we wrote the test like this:

[Test]\npublic void IsEven_ShouldNotThrow()\n{\n    var utils = new MathUtils();\n    Assert.DoesNotThrow(() => utils.IsEven(4));\n}

Coverage is still 100%, but the assert doesn't check the result! Now any mutant will survive. Mutation testing will immediately show that your tests are dummies.

Tools for Mutation Testing in C#

The main and de facto standard tool for .NET is Stryker.NET. It's an open-source framework that integrates easily into any C# project. It supports xUnit, NUnit, and MSTest.

Other tools (less popular):

\n
• VisualMutator — an outdated plugin for Visual Studio.
\n
• Nester