I recently gave a presentation to the test group at work that discusses how our project approaches testing. I’m going to be posting the sections from that presentation over the next month or so. It actually facilitated so much conversation that I only got through 2 of the 5 sections in twice the amount of time I initially set aside for the meeting. But that’s good, if people are talking about these process changes/improvements, they’re more likely to be utilized. Especially when those talking are managers :).

Anyway, over the next month or so I’ll post a new section once a week (maybe more often, no promises), covering what I’ve talked about. Hopefully I can get some good feedback from people here (if anyone even reads my blog!). I split it up into smaller chunks to reduce the size of the individual posts. These posts are to cover:

  • A Tester’s Role, through the entire lifecycle (3 separate posts)
  • Test Level Entry/Exit Criteria
  • System Test Procedure Generation
  • Test Case Input Generation

So here we go.

A Tester’s Role
The job of a tester is different at every stage in development, but the overall goal is to promote and facilitate quality. Traditionally, the role of tester was to click the button…after the system had already been implemented. Today, however, that alone just doesn’t cut it. To be effective, testers need to be involved in all stages of the project lifecycle (aka. testing throughout the lifecycle).

Different case studies have found that a bug found after release can cost 500 to 1500 times more to fix than if that same bug had been found at requirements. Obviously this implies that the bug was introduced during requirements, but this isn’t always the case and is beside the point. What if someone had been there to challenge what everyone assumed that particular requirement meant? What if they asked the tough and often times, obvious questions? Normally this is the job of the requirements analyst(s) on the project, but if you work on a smaller team like mine, there isn’t one. In this case, it must become the tester’s responsibility.

During Requirements
Requirements lay the foundation for everything the software is to become. If a foundation is weak, what happens to the building placed on it? Right, it either fails to be built at all, quickly crumbles to the ground, or costs tons of money to fix. A tester’s goal during requirements is to ask questions to get everyone on the same page. If two developers each make an assumption about a requirement, they could each make a different assumption. Worse yet, what if both of their assumptions are incorrect? Now we have two pieces of the system to fix instead of just one, not to mention that those pieces will likely be incompatible with each other to begin with (Continuous Integration can help with this problem, among other things).

An Example: We have a requirement which states: “The house shall have a front door.”

Is this requirement ambiguous? You bet it is! This is an easy one to make an assumption about, but what is the correct solution? Consider the following (yeah, yeah I know the pictures suck):

Each of these solutions is valid by the above requirement, but which is the correct solution? Even this one could be more correct, depending on the customer’s intentions:

The point is that based on the above requirement, all these solutions are valid. At this point, the tester can ask his/her self, how would I verify this? Just check if it has a door on the front of the house? But Where?

From a testing perspective, all requirements should be verifiable and quantifiable.

Who is qualified to answer this question? The architect? The homeowner? The contractor? The painter? Chances are that in this case the homeowner would have the final say (they’re paying for it aren’t they?), but what if they’re building the house for someone else? The requirement should be refined by the primary actor/actors in the use case(s) mapped to this requirement. In this case, it’s probably safe to say that the homeowner is the best choice (unless they really are building the house for someone else, in which case we would go to them). So, we contact the stakeholder and ask them to be more specific, and we get the following refined requirement.

The house shall have a front door on the first level.

Which is the first level?

The house shall have a front door on the ground floor of the building.

Where on the ground floor?

The house shall have a front door on the ground floor of the building, in between the two front windows.

And we have a winner!

This requirement appeared obvious at first (as so many do), but upon further inspection couldn’t have been much worse. As testers, requirements are our most precious artifact. They give us the means to verify everything that comes after it. In the absence of a dedicated requirements team or requirements analyst dedicated to our team, we need to make sure these requirements allow us to do our jobs. Ideally the developers shouldn’t have to guess even once during their implementation, but when they inevitably do they should know who to get in contact with to have their questions answered.