Testing multiple data sets with ScalaTest

10,775

Solution 1

There's a new feature for testing multiple data sets coming in ScalaTest 1.5, which you can currently try as a snapshot. It is released at scala-tools.org:

group id: org.scalatest artifact id: scalatest version: 1.5-SNAPSHOT

You mix in (or import the members of) TableDrivenPropertyChecks, then you can define tables like this:

val examples =
  Table(
    ("a", "b", "c", "d"),
    (  1,   1,   1,   1),
    (  1,   1,   1,   1),
    (  1,   1,   1,   1),
    (  1,   1,   1,   1)
  )

You pass a var arg list of tuples to Table. Each tuple has to have the same arity, in this case each tuple has arity 4 (4 members). The first tuple is all strings, and these define names for the columns. The subsequent tuples each define one row of data. You can put any type in the tuple, but in general each column would contain the same type. Though if you wanted you could have columns of type Any that can contain anything. You can have table with 1 to 22 columns. If you need more than 22 columns (the max tuple size in Scala is currently 22), what you could do is use a compound type in one or more columns.

Once you have a table, you can check them with forAll like this:

forAll (examples) { (a, b, c, d) =>
  a + b + c + d should equal (4)
}

forAll takes two parameter lists. The first is a table and the second is a "property function," which expresses something that should be true for every row of the table. forAll will take each row of the table (skipping the heading row of column names, of course) and make sure the property holds. If it doesn't, you get a nice error message saying which row of the table failed, what the values of the named columns were, etc.

A Table is a Seq of the data tuples, so you can also use it like a Seq. For example, you could get a Seq of Option[Exception] indicating which rows failed like this:

for ((a, b, c, d) <- examples) yield {
  failureOf { a + b + c + d should equal (4) }
}

The resulting Seq contains one Option for each row of data in the table, which is a None if the property passed for that row, and Some[Exception] if the property failed. The exception in the Some contains all the details about the failure.

Solution 2

Shared tests can be interesting to you. They allow you to define some set of tests like in this stack example:

  def nonEmptyStack(stack: Stack[Int], lastItemAdded: Int) {
    "be non-empty" in {
      assert(!stack.empty)
    }
    "return the top item on peek" in {
      assert(stack.peek === lastItemAdded)
    }
    "not remove the top item on peek" in {
      val size = stack.size
      assert(stack.peek === lastItemAdded)
      assert(stack.size === size)
    }
    "remove the top item on pop" in {
      val size = stack.size
      assert(stack.pop === lastItemAdded)
      assert(stack.size === size - 1)
    }
  }

and then in actual spec you can use it like this:

behave like nonEmptyStack(stackWithOneItem, lastValuePushed)

So in other words nonEmptyStack is parametrized set of tests that you can use with different data sets you want to test.

Solution 3

Another possibility for data-driven testing is to use the following syntax:

class SampleTest extends FunSuite {      
  val input = List((1, 1), (4, 2), (9, 3))

  input.foreach{i =>
    test(s"Test of math.sqrt(${i._1})") {
      assert(math.sqrt(i._1) === i._2)
    }
  }
}

Tests by default execute in parallel unless in build.sbt you set parallelExecution in Test := false.

10,775