Document

Report
Unit, Regression, and
Behavioral Testing
Jules White
[email protected]
Based On:
Unit Testing with JUnit and CUnit
by
Beth Kirby
[email protected]
206-685-0310
Dec 13, 2002
One minute summary
‘Code that isn’t tested doesn’t work’

Why test?
Code that isn’t regression tested suffers
from code rot (breaks eventually)

Why regression test?
A unit testing framework is unit &
regression testing on steroids
What is unit testing?
Testing is often divided into categories such as:

Unit testing
Testing an isolatable ‘unit’ of code, usually a class
e.g. AStack

Integration testing
Testing a module of code (e.g. a package)
Testing AStack + Analyzer, LStack + Analyzer

Application testing
Testing the code as the user would see it (black box)
Testing Delimiters
Writing a unit test - example
public void testAnalyzer() {
Analyzer analyzer = new Analyzer (….);
analyzer.process (“{“);
analyzer.process (“}”);
//checks to see if the delimiters matched
assertTrue (analyzer.match ());
}
What is a testing framework?
A test framework provides reusable test
functionality which:



Is easier to use (e.g. don’t have to write the
same code for each class)
Is standardized and reusable
Provides a base for regression tests
Why formalize unit testing?
Unit testing has often been done, but in
an ad hoc manner

E.g. adding a main method to a class,
which runs tests on the class
Axiom:


Code that isn’t tested doesn’t work
‘If code has no automated test case written
for it to prove that it works, it must be
assumed not to work.’ (Hightower and
Lesiecki)
Why use a testing framework?
Each class must be tested when it is
developed
Each class needs a regression test
Regression tests need to have standard
interfaces
Thus, we can build the regression test
when building the class and have a better,
more stable product for less work
Regression testing
New code and changes to old code can
affect the rest of the code base

‘Affect’ sometimes means ‘break’
I need to run tests on the old code, to
verify it works – these are regression
tests
Regression testing is required for a
stable, maintainable code base
Regression testing and
refactoring
‘Refactoring is a technique to restructure
code in a disciplined way.’ (Martin Fowler)
Refactoring is an excellent way to break
code.
Regression testing allows developers to
refactor safely – if the refactored code
passes the test suite, it works
Running automated tests
The real power of regression tests happens
when they are automated

This requires they report pass/fail results in a
standardized way
Can set up jobs to



Clean & check out latest build tree
Run tests
Put results on a web page & send mail (if tests fail)
JUnit & ant have code to do all of this
Some background
eXtreme programming (XP) is a
‘programming methodology that stresses
(among other things) testing and refactoring
The Apache community provides opensource software, including the Jakarta project
(server side java tools, e.g. tomcat (servlet
engine))
Ant is a build tool (like make)



Part of the Jakarta project
Is becoming the de facto standard for java projects
Is written in java, uses xml to specify build targets
Stateful Testing
What is Stateful Testing?
Example:
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(…..);
analyzer.process (“{“);
assertEqual (“{”, analyzer.getStack().topOfStack());
analyzer.process (“}”);
assertEqual( 0, analyzer.getStack().size() );
}
What is Wrong with Stateful
Testing?
What happens if Analyzer is refactored to
use a data structure other than a stack?
Example:
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(…..);
analyzer.process (“{“);
assertEqual (“{”, analyzer.getStack().topOfStack());
analyzer.process (“}”);
assertEqual( 0, analyzer.getStack().size() );
}
What is Wrong with Stateful
Testing?
Stateful testing does not allow the interface and
implementation to vary.
Stateful testing tightly couples the testing code
to the implementation so that refactoring breaks
the testing code
Stateful testing also prevents testing code to be
reused as new classes are added that
implement the same interface

}
e.g. a test targeted specifically for AStack cannot be
used by LStack
Behavioral Testing
Tests should check the expected behavior (input
/ output)
Tests should be coded to the specification not
the implementation
Example:
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(……);
analyzer.process (“{“);
analyzer.process (“}”);
Boolean expected = new Boolean(true);
expected.equals( analyzer.match () );
}
Mock Objects
How do you isolate an object for unit
testing when it relies on other objects?
Example:
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(……);
analyzer.process (“{“);
analyzer.process(“}”);
assertEqual( true, analyzer.match ()); // A stack is used here
}
Mock Objects
Mock Objects are used to stand in for the
real objects that the class relies on

e.g Stack is replaced by a MockStack
Mock Objects check to ensure that the
dependent object calls the correct
methods in the correct order
They can guarantee the input and output
to the dependent object and isolate the
source of errors
Mock Objects
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(new
MockObjectFactory());
…..
}
MockObjectFactory {
….
Stack make_stack () { return new MockStackObject ();}
}
Mock Objects
MockStackObject {
….
boolean popCalled = false;
boolean pushCalled = false;
boolean pushCalledFirst = false;
public void push (String str) {
pushCalled = true;
pushCalledFirst = !popCalled;
}
public String pop() {
popCalled = true;
return “{“;
}
Mock Objects
Example:
public void testAnalyzer() {
MockObjectFactory fact = new MockObjectFactory()
Analyzer analyzer = new Analyzer(fact);
analyzer.process (“{“);
analyzer.process (“}”);
assertTrue( fact.mockStack.pushCalledFirst());
assertTrue( fact.mockStack.popCalled ());
assertTrue( true, analyzer.match ()); // A MockStack is used here
}
What is Junit?
JUnit is a regression testing framework written
by Erich Gamma and Kent Beck
It is found at www.junit.org
It consists of classes that the developer can
extend to write a test – notably
junit.framework.TestCase - and related
structure to run and report the tests
Tool integration
Java IDEs

Eclipse – http://www.eclipse.org
Extensive support for JUnit


Netbeans http://www.netbeans.org/index.html
Idea - http://www.intellij.com/idea/
Writing a test
Directions can be found in the Junit
cookbook http://junit.sourceforge.net/doc/cookbook/cookbook.ht
m
Soundbite summary
1.
2.
3.
4.
Create an instance of TestCase:
Write methods which run your tests, calling them test<Foo>
Call a test runner with your test
When you want to check a value, call an assert method (e.g.
assertTrue()) and pass a condition that is true if the test
succeeds
Writing a test - example
public void testAnalyzer() {
Analyzer analyzer = new Analyzer(……);
analyzer.process (“{“);
analyzer.process (“}”);
Boolean expected = new Boolean(true);
expected.equals( analyzer.match () );
}
Specifying tests
JUnit



The framework will look in every class
given to it which implements TestCase
It uses reflection to find all methods that
start with ‘test’ (e.g. testMoneyMoneyBag)
It runs each such method as a test case
Test suites
In practice, you will want to run a group of
related tests (e.g. all the tests for a class)
To do so, group your test methods in a
class which extends TestCase
Override the constructor and (if desired)
the setUp and tearDown method
Test Suite example
Package test.mypackage;
import junit.framework.TestCase;
Import mypackage;
Public class MyClassTest extends TestCase {
public MyClassTest(String name) {
super(name);
public void setUp throws Exception {
// set up for each test
}
public void tearDown throws Exception {
// releases resources
}
public void testMyMethod throws Exception {
// Run tests
}
}
Test Suite methods
setUp – does initialization common to all
tests

It is run each time a test is run (can’t store
state in variables that are set in setUp)
tearDown – releases resources, e.g.
database connections.
Can also have other helper methods, just
don’t name them test<anything>
Running tests
Need to call a runner run method, e.g

junit.textui.TestRunner.run(suite());
Example main class for running tests
// imports
public class ATestSuite {
public static TestSuite suite () {
TestSuite suite = new TestSuite(“MyPackageTests”);
suite.addTestSuite(“MyClassTests.class”);
return suite;
}
public static void main(String args[]) {
junit.textui.TestRunner.run(suite());
}
}
Assert methods
There are a variety of assert methods
available, e.g.

AssertTrue, assertFalse, AssertNull,
assertSame, etc
In general, the syntax is


assert<Foo>(<type> expected, <type>
testVal)
Assert<Foo>(String message, <type>
expected, <type> testVal)
Project structure
One way to organize tests is to put them
in a parallel directory tree

E.g. MyPackage. …. Has a parallel tree of
tests.MyPackage. ….
Test suites can be collected, so


each class has a test class
each directory has test suite with a runner,
which runs the tests in that directory and
the suites in all subdirectories
Junit addons
Junit has a bunch of add on stuff that has been
written for it (your mileage may vary)
Examples
 Code generators – make guesses at what tests
are needed
 Load testers
 Code coverage
 Testers for thread safety
 ‘Helpful classes’ (e.g. RecursiveTestSuite is a
TestSuite that will recursively walk through the
directory structure looking for subclasses of
TestCase and will add them.)
Cactus, HttpUnit, Mock Objects
Cactus (from jakarta) is a simple test
framework for unit testing server-side java
code (Servlets, EJBs, Tag Libs, Filters, ...).
HttpUnit emulates the relevant portions of
browser behavior to allow automated testing
(on sourceforge)
Mock objects emulate objects from other
parts of the application, but have controlled
behaviour
Resources
Videos

http://video.google.com/videosearch?q=techtalks&pa
ge=1&lv=0&so=1
Web sites



Junit - http://www.junit.org
CppUnit - http://cppunit.sourceforge.net/
Ant - http://jakarta.apache.org/ant/index.html
Book

Java Tools for Extreme Programming: Mastering Open
Source Tools Including Ant, JUnit, and Cactus

similar documents