“Agile testing??? You mean as opposed to clumsy testing? “

As I think about, I realize that her comical comment is, in fact, accurate. The way I have to test on a traditional project is rather clumsy and inefficient compared to the way I test on an agile project. Piles of paperwork, hours logging issues in a tool, and slow turn-around time, just can’t compete with minimal paperwork, face to face collaboration, and fast turn-around times. I admit it. I am an agile proponent. I favor an environment where the management team, the development team and all of the stakeholders are on board with agile development. I have found that the team can produce systems not only faster, but of higher quality, when the team I am working with uses mature agile methods. But I am also a realist. Often I am faced with an environment where corporate policies and procedures prohibit the team from using agile methods “by the book.” I could choose to not work on those projects until the organization is fully agile compliant, but I find that there is value in helping software development teams “drive an agile peg in a CMMI hole.” If any of you have been there and done that, and have a few particularly useful and innovative tips to share, I’d love to hear from you.

There are requirements that should have been documented but weren’t. All requirements are imperfect and incomplete. Given this fact, programmers make assumptions, add behavior, get informal, undocumented clarification of requirements, and write a lot of code to implement business requirements that never made it to the requirements document.

A large percentage of production code has to do with technology and not business logic. This is the code that deals with inter-process communication, error detection and recovery, fault tolerance, GUI presentation, redo, undo, data formatting and conversion, interoperability with other production systems, etc.

Thus, black box testing alone will leave most of the code untested. A good code coverage tool will help the system tester identify untested lines of code and guide testers in the creation of the additional test cases needed to supplement their requirements based tests.

Short Answer: Because, typically, even a very comprehensive set of black box test cases only covers about 30%-40% of the logic in a program. If you are comfortable shipping a system where 60%-70% of the logic has not been tested by the system test team, then you don’t need a code coverage tool.

A manager I was talking wth recently lamented: “Our group is using agile techniques, but they aren’t working very well for the test effort. We are building the system in 2 week increments. Once the development team gets done with an increment, it is passed to us for system testing. The problem is that when we find bugs, we can’t get the development team to fix them because they are focused on the next increment, so either bugs build up, or the development team misses its goals for the next increment. This is causing a lot of process confusion for us. Any suggestions?”

Yes. I do have suggestions for this situation. The basic answer is that a team should not think of Agile as simply a series of short waterfalls. This automatically causes the problems described above. System testing must be integrated into an increment, not be a phase following development of the increment. During a 2 week iteration, everyone must collaborate on getting all the tasks for the increment done. This includes system testing. The increment is not done till the testing is done. If the finish date for an iteration arrives before the testing of a story is finished, then that story must be pulled from the increment and put back with the other unimplemented requirements. For a more complete and detailed answer, please listen to my webinar: How can a tester cope with a fast paced incremental delivery cycle