How to Break Software: A Practical Guide to Testing [ペーパーバック]

As a test designer, myself (and a competitor of Whittaker's) I can certainly find things to nitpick about this book. But I won't do that here, because the big picture is far more important. That picture is simply this: if you are confused about what to do to uncover problems in software before it ships, EVEN IF you have no specifications to test from and EVEN IF no one listens when you rant about "quality assurance processes" they should follow, then there are only a few testing books yet published that will help you. This is one of them.

The vulnerabilities are classified by a fault model, then the book systematically walks you through the procedures used to attack and break the software. Each vulnerability type is addressed:
User Interface
- inputs and outputs, with 6 attacks for breaking common input flaws and 4 for output flaws.
- data and computation, with 3 attacks against stored data and 3 against computation and feature interaction.

System Interface
- 3 media-based and 3 file-based attacks against the file system.
- how to test the application/operating system interface.

The book also comes with a Windows application that helps you to create the hostile environment with which to 'attack' the software being tested. Therein lies the sophistication of the book, which employs fault injection as a technique. This technique is not commonly used in any but the most advanced testing environments, which raises this book's credibility from ad hoc to a serious approach to software engineering. More importantly, it provides test professionals, especially those who are testing Windows applications, a catalog of common vulnerabilities to address. More importantly, it teaches test professionals to approach parts of the testing process from an exploitation point of view - after all, their job is to break the software.

My initial misgivings about this book vanished as soon as I started reading it, and were replaced by enthusiasm by the time I was finished. This book addresses a niche topic, but deserves a place in every software testing library.

* Apply inputs that force all the error messages to occur.
* Apply inputs that force the software to establish default values.
* Explore allowable character sets and data types.
* Overflow input buffers.
* Find inputs that may interact and test combinations of their values.
* Repeat the same input or series of inputs numerous times.
* Force different outputs to be generated for each input.
* Force invalid outputs to be generated.
* Force properties of an output to change.
* Force the screen to refresh.
* Apply inputs using a variety of initial conditions.
* Force a data structure to store too many or too few values.
* Investigate alternate ways to modify internal data constraints.
* Experiment with invalid operand and operator combinations.
* Force a function to call itself recursively.
* Force computation results to be too large or too small.
* Find features that share data or interact poorly.

The attacks for system interface are:

* Fill the file system to capacity.
* Force the media to be busy or unavailable.
* Damage the media.
* Assign an invalid file name.
* Vary file access permissions.
* Vary or corrupt file contents.

Each of the attacks is presented using the subsections:

* When to apply this attack.
* What software faults make this attack successful?
* How to determine if this attack exposes failures.
* How to conduct this attack.

This approach leads to a very thorough demonstration of how to perform rigorous software testing in a limited amount of time. If I ever teach a course in software testing, this is what I will use as a text.
The book includes a CD containing two software testing tools, one of which I wish was available when I was developing software. While it is running, you can move a slider to have it bind memory resources and learn the point of memory use where your software performance begins to suffer. This is very useful, and is much easier than trying to load up many other applications.
Software testing is a critical area of development that is still in the process of being codified into patterns for reuse. This book demonstrates many of the currently available strategies and should be read by all members of testing teams.