Chisel logo

Module 2.6: Testers2

Prev: Control Flow
Next: Sequential Logic

Motivation

The Chisel team has been working on an improved testing framework. Imaginatively titled "testers2", it provides the following improvements .

  • suitable for both unit tests and system integration tests
  • designed for composable abstractions and layering
  • highly usable, encouraging unit tests by making it as easy, painless (avoiding boilerplate and other nonsense), and useful as possible to write them

Planned

  • ablity to target multiple backends and simulators (possibly requiring a link to Scala, if the testvector is not static, or using a limited test constructing API subset, when synthesizing to FPGA)
  • will be included in base chisel3, to avoid packaging and dependency nightmares

Setup

In [ ]:
val path = System.getProperty("user.dir") + "/source/load-ivy.sc"
interp.load.module(ammonite.ops.Path(java.nio.file.FileSystems.getDefault().getPath(path)))
In [ ]:
import chisel3._
import chisel3.util._
import chisel3.experimental._
import chisel3.experimental.BundleLiterals._
import chisel3.tester._
import chisel3.tester.RawTester.test

This bootcamp requires some slight differences from the imports you might see elsewhere for chisel. The import chisel3.tester.RawTester.test brings in version of test(...) below that is designed specifically for the bootcamp


Basic Tester implementation

Testers2 starts with the same basic operations as iotesters. Here's a brief summary of the basic functionality mapping between the older iotesters and the new testers2

iotesters testers2
poke poke(c.io.in1, 6) c.io.in1.poke(6.U)
peek peek(c.io.out1) c.io.out1.peek(6.U)
expect expect(c.io.out1, 6) c.io.out1.expect(6.U)
step step(1) c.io.clock.step(1)
initiate Driver.execute(...) { c => test(...) { c =>

Let's start by looking at the simple pass through module from 2.1

In [ ]:
// Chisel Code, but pass in a parameter to set widths of ports
class PassthroughGenerator(width: Int) extends Module { 
  val io = IO(new Bundle {
    val in = Input(UInt(width.W))
    val out = Output(UInt(width.W))
  })
  io.out := io.in
}

Using the old style a simple test would look like this

val testResult = Driver(() => new Passthrough()) {
  c => new PeekPokeTester(c) {
    poke(c.io.in, 0)     // Set our input to value 0
    expect(c.io.out, 0)  // Assert that the output correctly has 0
    poke(c.io.in, 1)     // Set our input to value 1
    expect(c.io.out, 1)  // Assert that the output correctly has 1
    poke(c.io.in, 2)     // Set our input to value 2
    expect(c.io.out, 2)  // Assert that the output correctly has 2
  }
}
assert(testResult)   // Scala Code: if testResult == false, will throw an error
println("SUCCESS!!") // Scala Code: if we get here, our tests passed!
In [ ]:
test(new PassthroughGenerator(16)) { c =>
    c.io.in.poke(0.U)     // Set our input to value 0
    c.io.out.expect(0.U)  // Assert that the output correctly has 0
    c.io.in.poke(1.U)     // Set our input to value 1
    c.io.out.expect(1.U)  // Assert that the output correctly has 1
    c.io.in.poke(2.U)     // Set our input to value 2
    c.io.out.expect(2.U)  // Assert that the output correctly has 2
}

Just to illustrate the way testers2 advances the clock we can add some step operations to the previous examples.

In [ ]:
test(new PassthroughGenerator(16)) { c =>
    c.io.in.poke(0.U)     // Set our input to value 0
    c.clock.step(1)    // advance the clock
    c.io.out.expect(0.U)  // Assert that the output correctly has 0
    c.io.in.poke(1.U)     // Set our input to value 1
    c.clock.step(1)    // advance the clock
    c.io.out.expect(1.U)  // Assert that the output correctly has 1
    c.io.in.poke(2.U)     // Set our input to value 2
    c.clock.step(1)    // advance the clock
    c.io.out.expect(2.U)  // Assert that the output correctly has 2
}

What to notice in the above example

Testers2 test method requires a bit less boiler plate. What was the PeekPokeTester is now built into the process.

The poke and expect methods are now part of each individual io element. This gives important hints the the tester to make better checking of types. The peek and step operations are also now methods on io elements.

Another difference is that values poked and expected are Chisel literals. Although pretty simple here, it also provides stronger checking in more advanced and interesting examples. This will be further enhanced with coming improvements in the ability to specify Bundle literals

Modules with Decoupled Interfaces

In this section we will look at some of the tester2's tools for working with Decoupled interfaces. Decoupled takes a chisel data type and provides it with ready and valid signals. Testers2 provides some nice tools for automating and reliably testing these interfaces.

A queue example

The QueueModule passes through data whose type is determined by ioType. There are entries state elements inside the QueueModule meaning it can hold that many elements before it exerts backpressure.

In [ ]:
case class QueueModule[T <: Data](ioType: T, entries: Int) extends MultiIOModule {
  val in = IO(Flipped(Decoupled(ioType)))
  val out = IO(Decoupled(ioType))
  out <> Queue(in, entries)
}

Note the case class modifer is not generally required but seems to be in order for this example to be re-used in multiple cells in Jupyter

EnqueueNow and expectDequeueNow

testers2 has some built in methods for dealing with circuits with Decoupled interfaces in the IOs. In this example we will see how to insert and extract values from the queue.

method description
enqueueNow Add (enqueue) one element to a Decoupled input interface
expectDequeueNow Removes (dequeues) one element from a Decoupled output interface

Note: There is some required boiler plate initSource, setSourceClock, etc that is necessary to ensure that the ready and valid fields are all initialized correctly at the beginning of the test.

In [ ]:
test(QueueModule(UInt(9.W), entries = 200)) { c =>
    // Example testsequence showing the use and behavior of Queue
    c.in.initSource()
    c.in.setSourceClock(c.clock)
    c.out.initSink()
    c.out.setSinkClock(c.clock)
    
    val testVector = Seq.tabulate(200){ i => i.U }

    testVector.zip(testVector).foreach { case (in, out) =>
      c.in.enqueueNow(in)
      c.out.expectDequeueNow(out)
    }
}

EnqueueSeq and DequeueSeq

Now we are going to introduce two new methods that deal with enqueuing and dequeuing operations in single operations.

method description
enqueueSeq Continues to add (enqueue) elements from the Seq to a Decoupled input interface, one at a time, until the sequence is exhausted
expectDequeueSeq Removes (dequeues) elements from a Decoupled output interface, one at a time, and compares each one to the next element of the Seq

Note: The example below works fine but, as written, the enqueueSeq must finish before the expectDequeueSeq can begin. This example would fail if the testVector's size is made larger than the queue depth, because the queue would fill up and not be able to complete the enqueueSeq. Try it yourself to see what the failure looks like. In the next section we will show to construct this type of test properly.

In [ ]:
test(QueueModule(UInt(9.W), entries = 200)) { c =>
    // Example testsequence showing the use and behavior of Queue
    c.in.initSource()
    c.in.setSourceClock(c.clock)
    c.out.initSink()
    c.out.setSinkClock(c.clock)
    
    val testVector = Seq.tabulate(100){ i => i.U }

    c.in.enqueueSeq(testVector)
    c.out.expectDequeueSeq(testVector)
}

One more important take away from the last section is that the functions we just saw, enqueueNow, enqueueSeq, expectDequeueNow, and expectDequeueSeq are not complicated special case logic in testers2. Rather they are examples of the kinds of harness building that testers2 encourages you to build from the testers2 primitives. To see how these methods are implemented check out TestAdapters.scala

Fork and Join in testers2

In this section we will look at running sections of a unit test concurrently. In order to do this we will introduce two new features of testers2.

method description
fork launches a concurrent code block, additional forks can be run concurrently to this one via the .fork appended to end of the code block of the preceeding fork
join re-unites multiple related forks back into the calling thread

In the example below two forks are chained together, and then joined. In the first fork block the enqueueSeq will continue to add elements until exhausted. The second fork block will expectDequeueSeq on each cycle when data is available.

The threads created by fork are run in a deterministic order, largely according to their order as specified in code, and certain bug-prone operations that depend on other threads are forbidden with runtime checks.

In [ ]:
test(QueueModule(UInt(9.W), entries = 200)) { c =>
    // Example testsequence showing the use and behavior of Queue
    c.in.initSource()
    c.in.setSourceClock(c.clock)
    c.out.initSink()
    c.out.setSinkClock(c.clock)
    
    val testVector = Seq.tabulate(300){ i => i.U }

    fork {
        c.in.enqueueSeq(testVector)
    }.fork {
        c.out.expectDequeueSeq(testVector)
    }.join()
}

Using Fork and Join with GCD

In this section we will use the fork join methods to implement tests of Greatest Common Denominator GCD. Let's start by defining our IO bundles. We are going to add a bit of boiler plate here to allow us to use Bundle literals. As the comments say, it is hoped that we will soon have support for autogeneration of the literal support code.

In [ ]:
class GcdInputBundle(val w: Int) extends Bundle {
  val value1 = UInt(w.W)
  val value2 = UInt(w.W)
}
In [ ]:
class GcdOutputBundle(val w: Int) extends Bundle {
  val value1 = UInt(w.W)
  val value2 = UInt(w.W)
  val gcd    = UInt(w.W)
}

Now let's look at a Decoupled version of GCD. We've modified it a bit here to use the Decoupled wrapper that adds a ready and a valid signal to the input and output Bundle. The Flipped wrapper takes the Decoupled GcdInputBundle which by default is created as an output and converts each field to the opposite direction (recursively). The data elements of the bundled arguments to Decoupled are placed in the top level field bits.

In [ ]:
/**
  * Compute GCD using subtraction method.
  * Subtracts the smaller of registers x and y from the larger until register y is zero.
  * value input register x is then the Gcd
  * returns a packet of information with the two input values and their GCD
  */
class DecoupledGcd(width: Int) extends MultiIOModule {

  val input = IO(Flipped(Decoupled(new GcdInputBundle(width))))
  val output = IO(Decoupled(new GcdOutputBundle(width)))

  val xInitial    = Reg(UInt())
  val yInitial    = Reg(UInt())
  val x           = Reg(UInt())
  val y           = Reg(UInt())
  val busy        = RegInit(false.B)
  val resultValid = RegInit(false.B)

  input.ready := ! busy
  output.valid := resultValid
  output.bits := DontCare

  when(busy)  {
    // during computation keep subtracting the smaller from the larger
    when(x > y) {
      x := x - y
    }.otherwise {
      y := y - x
    }
    when(y === 0.U) {
      // when y becomes zero computation is over, signal valid data to output
      output.bits.gcd := x
      output.bits.value1 := xInitial
      output.bits.value2 := yInitial
      output.bits.gcd := x
      output.valid := true.B
      busy := false.B
    }
  }.otherwise {
    when(input.valid) {
      // valid data available and no computation in progress, grab new values and start
      val bundle = input.deq()
      x := bundle.value1
      y := bundle.value2
      xInitial := bundle.value1
      yInitial := bundle.value2
      busy := true.B
      resultValid := false.B
    }
  }
}

Our test looks pretty much the same as the earlier Queue tests. But there's more going on because the computation take multiple cycles so the input enqueue process is blocked as each GCD is computed. The good news is that test side of this is simple and consistent across different Decoupled circuits.

Also introduced here is the new Chisel3 Bundle literal notation. Consider the line

new GcdInputBundle(16)).Lit(_.value1 -> x.U, _.value2 -> y.U)

GcdInputBundle defined above has two fields value1 and value2. We create a bundle literal by first creating a bundle then calling its .Lit method. That method takes a variable argument list of key/value pairs, where the key (e.g. _.value) is the field name and the value (e.g. x.U) is a chisel hardware literal, the Scala Int x is converted into a Chisel UInt literal. The _. in front of the field name is necessary to bind the name value to the bundle internals.

This may not be the perfect notation but in extensive development discussions it was viewed as the best balance between minimizing boilerplate and the notational limitations available in Scala.

In [ ]:
test(new DecoupledGcd(16)) { dut =>
  dut.input.initSource().setSourceClock(dut.clock)
  dut.output.initSink().setSinkClock(dut.clock)

  val testValues = for { x <- 1 to 10; y <- 1 to 10} yield (x, y)
  val inputSeq = testValues.map { case (x, y) =>
    (new GcdInputBundle(16)).Lit(_.value1 -> x.U, _.value2 -> y.U)
  }
  val resultSeq = testValues.map { case (x, y) =>
    new GcdOutputBundle(16).Lit(_.value1 -> x.U, _.value2 -> y.U, _.gcd -> BigInt(x).gcd(BigInt(y)).U)
  }

  fork {
    dut.input.enqueueSeq(inputSeq)
  }.fork {
    dut.output.expectDequeueSeq(resultSeq)
  }.join()
}

You're done!

Return to the top.