B.4.1 Use integration tests for your data layer
How should you test your data layer? Should you abstract away the
database interfaces? Should you use the real database?
I usually write integration-style tests for the data layer (the part of the
app structure that talks directly to the database) in my applications
because data logic is almost always divided between the application
logic and the database itself (triggers, security rules, referential integ-
rity, and so on). Unless you can test the database logic in complete iso-
lation (and I’ve found no really good framework for this purpose), the
only way to make sure it works in tests is to couple testing the data-
layer logic to the real database.
Testing the data layer and the database together leaves few surprises
for later in the project. But testing against the database has its prob-
lems, the main one being that you’re testing against state shared by
many tests. If you insert a line into the database in one test, the next
test can see that line as well.
What we need is a way to roll back the changes we make to the data-
base, and thankfully there’s good support for this in the current test
tools and the .NET framework.
B.4.2 Use rollback attributes
The three major frameworks—MbUnit, NUnit, and xUnit—support a
attribute that you can put on top of your test
method. When used, the attribute creates a special database transac-
tion that the test code runs in. When the test is finished, the database
transaction is rolled back automatically, and the changes to the data-
To learn more about how this works, see an MSDN article I wrote a
while back, called “Simplify Data Layer Unit Testing using Enterprise
Services” at http://msdn.microsoft.com/en-us/magazine/cc163772.aspx.
If you aren’t interested in using the
attributes the frame-
works provide, you can use the simple class introduced in .NET 2.0
APPENDIX B Extra tools and frameworks
B.4.3 Use TransactionScope to roll back
For examples on how to use the
class in your setup
and teardown code, see a blog article called “Even Simpler Database
Unit Testing with TransactionScope” at http://www.bbits.co.uk/blog/
Some feel that another good option is to run the tests against an in-
memory database. My feelings on that are mixed. On the one hand, it’s
closer to reality, in that you also test the database logic. On the other
hand, if your application uses a different database engine, with differ-
ent features, there’s a big chance that some things will pass or fail dur-
ing tests with the in-memory database, and will work differently in
production. I choose to work with whatever is as close to the real thing
as possible. Usually that means using the same database engine.
“How do I test my web pages?” is another question that comes up a lot.
Here are some tools that can help you in this quest:
Team System Web Test
The following is a short description of each tool.
Ivonna is a unit-testing framework that abstracts away the need to run
ASP.NET-related tests using a real HTTP session and pages. It does
some powerful things behind the scenes, such as compiling pages that
you want to test and letting you test controls inside them without need-
ing a browser session, and it fakes the full HTTP runtime model.
You write the code in your unit tests just like you’re testing other in-
memory objects. There’s no need for a web server and such nonsense.
B.5 Web testing
Ivonna is being developed in partnership with Typemock and runs as
an add-on to the Typemock Isolator framework. You can get Ivonna at
B.5.2 Team System Web Test
Visual Studio Team Test and Team Suite editions include the powerful
ability to record and replay web requests for pages and verify various
things during these runs. This is strictly integration testing, but it’s
really powerful. The latest versions also support recording Ajax actions
on the page, and make things much easier to test in terms of usability.
You can find more info on Team System at http://msdn.microsoft.com/
NUnitAsp is an open source framework that’s no longer being sup-
ported but is still used in many places. It allows you to write tests
against live HTTP page objects.
Most people end up using NUnitAsp for acceptance testing. In those
cases, it would be better to use frameworks such as Watir and WatiN,
You can get NUnitAsp at http://nunitasp.sourceforge.net/.
Watir (pronounced “water”) stands for “Web application testing in
Ruby”. It’s open source, and it allows scripting of browser actions
using the Ruby programming language. Many Rubyists swear by it,
but it does require that you learn a whole new language.
You can get Watir at http://wtr.rubyforge.org/.
WatiN (pronounced “what-in”) is a product inspired by Watir. You
don’t need to know Ruby to use WatiN, but it offers much the same
scripting abilities as Watir.
You can get WatiN at http://watin.sourceforge.net/.
APPENDIX B Extra tools and frameworks
Selenium is a suite of tools designed to automate web app testing across
many platforms. It has existed longer than all the other frameworks in
this list, and it also has an API wrapper for .NET.
Selenium is an integration testing framework, and it’s in wide use. It’s a
good place to start. But beware: it has many features and the learning
curve is high.
You can get it at http://selenium.openqa.org/.
UI testing is always a difficult task. I’m not a great believer in writing
unit tests or integration tests for UIs because the return on such tests is
low compared to the amount of time you invest in writing them. UIs
change too much to be testable in a consistent manner, as far as I’m
concerned. That’s why I usually try to separate all the logic from the
UI into a lower layer that I can test separately with standard unit-test-
Nevertheless, there are several tools that try to make the UI-testing job
Team System UI Tests
Here is a short rundown of each tool.
NUnitForms is an open source framework that allows you to instanti-
ate Windows Forms classes and check values of controls inside them. It
has specific APIs for getting controls in a form and asserting values
against them, but it will only work for simple controls.
More complex controls aren’t supported and the framework doesn’t
seem to be in active development anymore, so I recommend not using
B.6 UI testing
it. Instead, I recommend Project White (discussed next) for WinForm
You can get it at http://nunitforms.sourceforge.net/.
B.6.2 Project White
Project White is, in a way, a successor to NUnitForms, in that it sup-
ports a richer set of application types (WPF, WinForm, Win32, and
Java JWT), and sports a newer API with better usability. Unlike
NUnitForms, White is more of an integration-test framework, because
it allows spawning separate processes that it tests against.
White uses the UIAutomation API (part of Windows) to do its bid-
ding, which gives it much more power. You can think of it as Selenium
or WatiN for WinForms.
You can get White at http://www.codeplex.com/white.
B.6.3 Team System UI Tests
The upcoming version of Visual Studio Team System will support a
new kind of test—a UI test. You’ll be able to record actions on UI win-
dows and play them back and verify assertions during test runs. As
with all Team System tools, its main advantage will be its integration
with other Team System tools, reports, source control, and servers.
You can learn more about Team System at http://msdn.microsoft.com/
Threads have always been the bane of unit testing. They’re simply untest-
able. That’s why new frameworks are emerging that let you test thread-
related logic (deadlocks, race conditions, and so on), such as these:
I’ll give a brief rundown of each tool.
B.7 Thread-related testing
APPENDIX B Extra tools and frameworks
B.7.1 Typemock Racer
Typemock Racer is a managed framework for multithreaded code test-
ing that helps visualize, detect, and resolve deadlocks and race condi-
tions in managed code. You use it by putting an attribute on top of an
existing test method, and the engine does all the work. It also allows
full thread visualization during debugging of a threaded test.
Full disclosure: during the writing of this book, I’ve been part of the
developer team at Typemock.
Racer is a commercial product that works with all flavors of Visual
Studio (including Express) and all test frameworks. You can get it at
B.7.2 Microsoft CHESS
CHESS is an upcoming tool that will be offered by Microsoft with
Visual Studio 2010. (It’s currently offered only as part of MS
Research.) Much like Typemock Racer, CHESS attempts to find
thread-related problems (deadlocks, hangs, livelocks, and more) in
your code by running all relevant permutations of threads on existing
code. These tests are written as simple unit tests.
CHESS currently supports native code, but a managed .NET version
should be coming out soon as part of Visual Studio Team System (a
You can get CHESS at
This is a little open source framework I developed a while back. It
allows you to run multiple threads during one test to see if anything
weird happens to your code (deadlocks, for example). It isn’t feature-
complete, but it’s a good attempt at a multithreaded test (rather than a
test for multithreaded code).
You can get it from my blog, at http://weblogs.asp.net/rosherove/
Acceptance tests enhance collaboration between customers and devel-
opers in software development. They enable customers, testers, and
programmers to learn what the software should do, and they automati-
cally compare that to what it actually does. They compare customers'
expectations to actual results. It’s a great way to collaborate on compli-
cated problems (and get them right) early in development.
Unfortunately, there are few frameworks for automated acceptance
testing, and just one that works these days! I’m hoping this will change
soon. Here are the tools we’ll look at:
Let’s take a closer look.
FitNesse is a lightweight, open source framework that makes it easy
for software teams to define acceptance tests—web pages containing
simple tables of inputs and expected outputs—and to run those tests
and see the results.
FitNesse is quite buggy, but it has been in use in many places with
varying degrees of success.
You can learn more about FitNesse at www.fitnesse.org.
StoryTeller is a response to FitNesse, which has existed for a long time
but presented many usability and stability problems for users. It’s an
open source framework, still in development, that attempts to create a
more compelling UI and usability story than FitNesse. There aren’t
many details on it yet, but hopefully it will be out soon. It will support
running tests written for FitNesse.
The project page is at http://storyteller.tigris.org/, but most of the real
details are found on the author’s blog: http://codebetter.com/blogs/jer-
B.8 Acceptance testing
.NET 21, 25, 56, 69, 100, 151
extension methods 129, 269
Moq framework 100, 126
Rhino Mocks 100
.NET mock object frameworks 129
AAA-style stubs 128
abstract class 259
abstract test class 159
abstract test driver class pattern 159
abstract test infrastructure class pattern 152
acceptance testing 268, 283
acceptance-style test 251
accidental bugging 9
Act action 30
action attributes 34
action-driven testing 83
Adapt Parameter 96
Addison-Wesley 227, 231
agent of change 220
agile 142, 234
analyzer object 188
annotated method 79
anonymous delegate 120, 124–125
anonymous method 124, 249
constrained test order 192
external-shared-state corruption 198
hidden test call 194
shared-state corruption 196
API changes 174
Arrange action 30
arrange-act-assert model 103, 126
arrange-act-assert style 269
arrange-act-assert syntax 269
art of unit testing 52
ASP.NET 274, 278
Assert 31, 35–36, 38, 42–43
Assert action 30
Assert class 31
assert failures 180
assert messages 209, 212
best practices 212
assert utility class 168
assert utility method 168
assertion logic 188
assumptions 205, 209
Documents you may be interested
Documents you may be interested