Test parameterization in xUnit.net similar to NUnit

Question

Are there any means in xUnit.net framework similar to the following features of NUnit?

[Test, TestCaseSource("CurrencySamples")]
public void Format_Currency(decimal value, string expected){}

static object[][] CurrencySamples = new object[][]
{
    new object[]{ 0m, "0,00"},
    new object[]{ 0.0004m, "0,00"},
    new object[]{ 5m, "5,00"},
    new object[]{ 5.1m, "5,10"},
    new object[]{ 5.12m, "5,12"},
    new object[]{ 5.1234m, "5,12"},
    new object[]{ 5.1250m, "5,13"}, // round
    new object[]{ 5.1299m, "5,13"}, // round
}

This will generate 8 separate tests in NUnit GUI

[TestCase((string)null, Result = "1")]
[TestCase("", Result = "1")]
[TestCase(" ", Result = "1")]
[TestCase("1", Result = "2")]
[TestCase(" 1 ", Result = "2")]
public string IncrementDocNumber(string lastNum) { return "some"; }

This will generate 5 separate tests and automatically compare the results (Assert.Equal()).

[Test]
public void StateTest(
    [Values(1, 10)]
    int input,
    [Values(State.Initial, State.Rejected, State.Stopped)]
    DocumentType docType
){}

This will generate 6 combinatorial tests. Priceless.

Few years ago I tried xUnit and loved it but it lacked these features. Can't live without them. Has something changed?

Solution 1

xUnit offers a way to run parameterized tests through something called data theories. The concept is equivalent to the one found in NUnit but the functionality you get out of the box is not as complete.

Here's an example:

[Theory]
[InlineData("Foo")]
[InlineData(9)]
[InlineData(true)]
public void Should_be_assigned_different_values(object value)
{
    Assert.NotNull(value);
}

In this example xUnit will run the Should_format_the_currency_value_correctly test once for every InlineDataAttribute each time passing the specified value as argument.

Data theories are an extensibility point that you can use to create new ways to run your parameterized tests. The way this is done is by creating new attributes that inspect and optionally act upon the arguments and return value of the test methods.

You can find a good practical example of how xUnit's data theories can be extended in AutoFixture's AutoData and InlineAutoData theories.

Solution 2

Let me throw one more sample here, just in case it saves some time to someone.

[Theory]
[InlineData("goodnight moon", "moon", true)]
[InlineData("hello world", "hi", false)]
public void Contains(string input, string sub, bool expected)
{
    var actual = input.Contains(sub);
    Assert.Equal(expected, actual);
}

Solution 3

On your first request, you can follow the examples found here.

You can construct a static class containing the data necessary for a collection of tests

using System.Collections.Generic;

namespace PropertyDataDrivenTests
{
    public static class DemoPropertyDataSource
    {
        private static readonly List<object[]> _data = new List<object[]>
            {
                new object[] {1, true},
                new object[] {2, false},
                new object[] {-1, false},
                new object[] {0, false}
            };

        public static IEnumerable<object[]> TestData
        {
            get { return _data; }
        }
    }
}

Then, using the MemberData attribute, define the test as such

public class TestFile1
{
    [Theory]
    [MemberData("TestData", MemberType = typeof(DemoPropertyDataSource))]
    public void SampleTest1(int number, bool expectedResult)
    {
        var sut = new CheckThisNumber(1);
        var result = sut.CheckIfEqual(number);
        Assert.Equal(result, expectedResult);
    }
}

or if you're using C# 6.0,

[Theory]
[MemberData(nameof(PropertyDataDrivenTests.TestData), MemberType = typeof(DemoPropertyDataSource))]

The first argument of MemberDataAttribute allows you to define the member you use as a datasource, so you have a fair amount of flexibility on reuse.

Solution 4

According to this article in xUnit you have three "parametrization" options:

1. InlineData
2. ClassData
3. MemberData

InlineData example

[Theory]
[InlineData(1, 2)]
[InlineData(-4, -6)]
[InlineData(2, 4)]
public void FooTest(int value1, int value2)
{
    Assert.True(value1 + value2 < 7)
}

ClassData example

public class BarTestData : IEnumerable<object[]>
{
    public IEnumerator<object[]> GetEnumerator()
    {
        yield return new object[] { 1, 2 };
        yield return new object[] { -4, -6 };
        yield return new object[] { 2, 4 };
    }

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}


[Theory]
[ClassData(typeof(BarTestData))]
public void BarTest(int value1, int value2)
{
    Assert.True(value1 + value2 < 7)
}

MemberData example

[Theory]
[MemberData(nameof(BazTestData))]
public void BazTest(int value1, int value2)
{
    Assert.True(value1 + value2 < 7)
}

public static IEnumerable<object[]> BazTestData => new List<object[]>
    {
        new object[] { 1, 2 },
        new object[] { -4, -6 },
        new object[] { 2, 4 },
    };

Solution 5

I found a library that produces equivalent functionality to NUnit's [Values] attribute called Xunit.Combinatorial:

It allows you to specify parameter-level values:

[Theory, CombinatorialData]
public void CheckValidAge([CombinatorialValues(5, 18, 21, 25)] int age, 
    bool friendlyOfficer)
{
    // This will run with all combinations:
    // 5  true
    // 18 true
    // 21 true
    // 25 true
    // 5  false
    // 18 false
    // 21 false
    // 25 false
}

Or you can implicitly have it figure out the minimal number of invocations to cover all possible combinations:

[Theory, PairwiseData]
public void CheckValidAge(bool p1, bool p2, bool p3)
{
    // Pairwise generates these 4 test cases:
    // false false false
    // false true  true
    // true  false true
    // true  true  false
}

View more solutions

Comments

[Sergii Volchkov]

Apparently, it is not allowed to use decimal literals as attribute parameters.

[MsBao]

@RubenBartelink your link is not found. Go here instead: blog.benhall.me.uk/2008/01/introduction-to-xunit-net-extensi‌​ons

[Daniel A.A. Pelsmaeker]

You'll need the xUnit.net: Extensions (NuGet Package) or otherwise the [Theory] attribute is not available.

[Isaac Kleinman]

It would be great if the most-recommended .NET unit testing framework had some documentation..

[Isaac Kleinman]

Strangely, the test runner counts multiple test cases as one test.

[Stephen Zeng]

The code in the answer cannot compile since decimal is not allowed as attribute parameters in C#. I strongly urge the author to update the answer as it is misleading.

[Michał Poreda]

@StephenZeng The actual values used in the test are besides the point. Nonetheless, the code in the example should always compile unless stated otherwise. Good catch :)

[Stephen Zeng]

@Enrico Campidoglio I certainly see your point is about data theory, however, different people may see it from different angels. Glad you have updated it : )

[nathanchere]

Google says your SO answers ARE the xUnit documentation.

[Matthew]

@SergiiVolchkov You may be able to use decimals the way NUnit handles TestCase by enclosing the value in quotes.

[Tore Aurstad]

Interesting choice of words, but your answer helped me out adjust my NUnit to nomenclature to corresponding XUnit nomenclature. Thanks. XUnit is de facto default testing framework other than MsTest in Asp.net Core 2?

[Brett Rowberry]

This is lovely.