Isolation (mock object) frameworks
5.11.2 Verifying the wrong things
Mock objects allow us to verify that methods were called on our inter-
faces, but that doesn’t necessarily mean that we’re testing the right thing.
Testing that an object subscribed to an event doesn’t tell us anything
about the functionality of that object. Testing that when the event is
raised something meaningful happens is a better way to test that object.
5.11.3 Having more than one mock per test
It’s considered good practice to only test one thing per test. Testing
more than one thing can lead to confusion and problems maintaining
the test. Having two mocks in a test is the same as testing several
things. If you can’t name your test because it does too many things, it’s
time to separate it into more than one test.
5.11.4 Overspecifying the tests
If your test has too many expectations, you may create a test that
breaks down with even the lightest of code changes, even though the
overall functionality still works. Consider this a more technical way of
not verifying the right things. Testing interactions is a double-edged
sword: test it too much, and you start to lose sight of the big pic-
ture—the overall functionality; test it too little, and you’ll miss the
important interactions between objects.
Here are some ways to balance this effect:
Use nonstrict mocks when you can.
The test will break less often because of unexpected method calls.
This helps when the private methods in the production code keep
Use stubs instead of mocks when you can.
You only need to test one scenario at a time. The more mocks you
have, the more verifications will take place at the end of the test, but
only one of them will usually be the important one. The rest will be
noise against the current test scenario.
Avoid using stubs as mocks.
Use a stub only for faking return values into the program under test,
or to throw exceptions. Don’t verify that methods were called on
stubs. Use a mock only for verifying that some method was called on
it, but don’t use it to return values into your program under test. If
you can’t avoid this situation, you should probably be using a stub
and testing something other than what the mock object receives.
Don’t repeat logic in your tests.
If you’re asserting that some calculation is correct in your code,
make sure your test doesn’t repeat the calculation in the test code, or
the bug might be duplicated and the test will magically pass.
Don’t use “magic” values.
Try to always use hardcoded, known return values to assert against
production code, and don’t create expected values dynamically. That
would significantly increase the chances for an unreadable test or a
bug in the test.
Overspecification is a common form of test abuse. Make sure you keep
your eyes on this by doing frequent test reviews with your peers.
Dynamic mock objects are pretty cool, and you should learn to use
them at will. But it’s important to lean toward state-based testing (as
opposed to interaction testing) whenever you can, so that your tests
assume as little as possible about internal implementation details.
Mocks should be used only when there’s no other way to test the
implementation, because they eventually lead to tests that are harder to
maintain if you’re not careful.
Learn how to use the advanced features of an isolation framework such
as Rhino Mocks or Typemock Isolator, and you can pretty much make
sure that anything happens or doesn’t happen in your tests. All you
need is for your code to be testable.
You can also shoot yourself in the foot by creating overspecified tests
that aren’t readable or will likely break. The art lies in knowing when
Isolation (mock object) frameworks
to use dynamic versus handwritten mocks. My guideline is that, when
the code using the isolation framework starts to look ugly, it’s a sign
that you may want to simplify things. Use a handwritten mock, or
change your test to test a different result that proves your point but is
easier to test.
When all else fails and your code is hard to test, you have three choices:
use a “super” framework like Typemock Isolator, change the design, or
quit your job.
Isolation frameworks can help make your testing life much easier and
your tests more readable and maintainable. But it’s also important to
know when they might hinder your development more than they help.
In legacy situations, for example, you might want to consider using a
different framework based on its abilities. It’s all about picking the
right tool for the job, so be sure to look at the big picture when consid-
ering how to approach a specific problem in testing.
That’s it! We’ve covered the core techniques for writing unit tests. The
next part of the book deals with managing test code, arranging tests,
and patterns for tests that you can rely on, maintain easily, and under-
The test code
his part of the book covers techniques for managing and organizing
unit tests and for ensuring that the quality of unit tests in real-world
projects is high.
Chapter 6 first covers the role of unit testing as part of an automated
build process, and follows with several techniques for organizing the
different kinds of tests according to categories (speed, type) with a goal
of reaching what I call the “safe green zone.” It also explains how to
“grow” a test API or test infrastructure for your application.
In chapter 7, we’ll take a look at the three basic pillars of good unit
tests—readability, maintainability, and trustworthiness—and look at
techniques to support them. If you only read one chapter in the book,
this should be it.
Test hierarchies and
This chapter covers
•Running unit tests during automated nightly builds
•Using continuous integration for automated builds
•Organizing tests in a solution
•Exploring test class inheritance patterns
nit tests are as important to an application as the production source code.
As with the regular code, you need to give careful thought to where the
tests reside, both physically and logically, in relation to the code under
test. If you put the tests in the wrong place, the tests you’ve written so
carefully may not be run.
Similarly, if you don’t devise ways to reuse parts of your tests, create util-
ity methods for testing, or use test hierarchies, you’ll end up with test
code that’s either unmaintainable or hard to understand.
This chapter addresses these issues with patterns and guidelines that will
help you shape the way your tests look, feel, and run, and will affect how
well they play with the rest of your code and with other tests.
Where the tests are located depends on where they will be used and who
will run them. There are two common scenarios: tests run as part of the
automated build process, and tests run locally by developers on their own
CHAPTER 6 Test hierarchies and organization
machines. The automated build process is very important, and that’s
what we’ll focus on.
The power of the automated build cannot and should not be ignored. If
you plan to make your team more agile and equipped to handle
requirement changes as they come into your shop, you need to be able
to do the following:
Make a small change to your code.
Run all the tests to make sure you haven’t broken any existing func-
Make sure your code can still integrate well and not break any other
projects you depend upon.
Running those tests lets you know whether you’ve broken any existing
or new functionality.
your code with the other projects will
indicate whether or not you broke the compilation of the code or things
that are logically dependent on your code.
Integrating your code usually means doing the following:
Getting the latest version of everyone’s source code from the source
Trying to compile it all locally
Running all tests locally
Fixing anything that has been broken
Checking in your source code
An automated build process combines all these steps under a special
build script that will make sure all these things are done without
human interaction. If anything breaks in the process, the build server
will notify the relevant parties of a
6.1.1 Anatomy of an automated build
An automated build process should perform
the bold points in
the following list, but it may include many other things:
Get the latest version of all projects in question.
6.1 Having automated builds run automated tests
Having automated builds run automated tests
Compile all the projects in their latest version.
Deploy build output to a test server.
Run tests locally or on the test server.
Create an archive of build outputs based on date and build number.
Deploy outputs to staging or even production server.
Configure and install components on target server.
Notify relevant people (by email) if any of the steps failed.
Create reports on build quality, history, and test statuses.
Create tasks or work items automatically (such as adding a Team
System work item) if specific tasks have failed.
The easiest way to get an automated build going is by creating a build
process and scripts as soon as the project is started. It’s much easier to
create an automated build for a small project and keep adding to it as
the project grows than it is to start later in the game.
There are many tools that can help you create an automated build sys-
tem. Some are free or open source, and some are commercial. Here are
a few tools you can look at:
Visual Build Pro (www.kinook.com)
Visual Studio Team Foundation Server (http://msdn.microsoft.com/
These are all configuration-based programs that allow you to create a
series of steps that will be run in a hierarchy structure. You can create
custom commands to be run, and you can schedule these builds to run
CHAPTER 6 Test hierarchies and organization
6.1.2 Triggering builds and continuous integration
is literally about making the automated
build and integration process run continuously. For example, you
could have the build run every time someone checks in source code to
the system, or every 45 minutes.
One popular continuous integration tool is CruiseControl.NET. It’s
fully open source and supports both the idea of
, which are individ-
ual commands that are run during a build, and the concept of
which can start a build automatically when certain events occur, such
as source control updates.
Among the commercial tools, Visual Studio Team System 2008 sup-
ports automated builds and continuous integration out of the box. If
that’s a bit beyond your budget, look at FinalBuilder and Visual Build
Pro. These two commercial and highly successful build tools allow
visual editing and maintenance of automated build projects. That
means easier maintenance of the build file, which can get pretty scary
for larger projects.
6.1.3 Automated build types
You can configure many types of automated builds to produce differ-
ent results or builds that run in specific amounts of time (all of which
compile the code first, though). Here are a few examples:
A nightly build
runs all the long-running tests.
runs system tests.
A release build
runs the nightly build.
deploys to server and archives.
A CI (continuous integration) build
runs all the fast-running tests.
finishes in less than 10 minutes.
When you start writing tests, you should categorize them by their run-
Documents you may be interested
Documents you may be interested