Life, Teams, and Software Engineering

Month: April 2009

Software Testing: A Tester’s Role, Part 2

During Implementation
A tester’s role during implementation depends on how close the tester wants to be to the actual implementation of the system under test (SUT). If they are meant to be an independent tester, then treat the code as your Kryptonite. Don’t go near it! The less you know about how it’s implemented the better. Ideally you should know nothing, but depending upon your operating environment that just might not be possible. At the very least, you had better be able to forget about the how when you go to verify that the system does what it’s supposed to.

During the implementation phase of the project, the test team can take a built version of the system and exercise it inside its intended operating conditions. Testing should be performed from the perspective of all human actors in the defined use cases. This protects against the team concentrating too hard on one actor’s perspective. These “stagings” can be performed weekly, bi-weekly, monthly, or even daily if they so choose. How often depends on how quickly we want/need to receive feedback.

Hopefully the developers are performing test-first development (for all our sakes), but if they aren’t the test team will need to be extra vigilant during these stagings because they are effectively performing the unit and unit integration testing.

An added benefit to staging is that by the time the system tests roll around the test team should be well versed in the system. They should know what it’s intended to do, how to make it do those things, and what caused the most frequent failures in the past.

A note on communication. Test plans and procedures should be “public domain” on your project. Everyone should be able to access them at any time, specifically the development team. If the developers know how you intend to exercise the system, they can make sure the system works in those cases. Remember that the point isn’t to catch the mistakes of the developers to humilitate them. There is no line drawn in the sand, we’re on the same team! The point is to catch problems across the board to protect the project and the people working on it.

The sooner we catch these problems, the easier they are to fix.
The easier they are to fix, the cheaper they are.
The cheaper they are, the less overhead we incur.
The less overhead we incur, the more work we can perform with that excess.

Or we could just take that money to the bank. Either way, less money spent can be allocated elsewhere.

Software Testing: A Tester’s Role, The Prelude

I recently gave a presentation to the test group at work that discusses how our project approaches testing. I’m going to be posting the sections from that presentation over the next month or so. It actually facilitated so much conversation that I only got through 2 of the 5 sections in twice the amount of time I initially set aside for the meeting. But that’s good, if people are talking about these process changes/improvements, they’re more likely to be utilized. Especially when those talking are managers :).

Anyway, over the next month or so I’ll post a new section once a week (maybe more often, no promises), covering what I’ve talked about. Hopefully I can get some good feedback from people here (if anyone even reads my blog!). I split it up into smaller chunks to reduce the size of the individual posts. These posts are to cover:

  • A Tester’s Role, through the entire lifecycle (3 separate posts)
  • Test Level Entry/Exit Criteria
  • System Test Procedure Generation
  • Test Case Input Generation

So here we go.

A Tester’s Role
The job of a tester is different at every stage in development, but the overall goal is to promote and facilitate quality. Traditionally, the role of tester was to click the button…after the system had already been implemented. Today, however, that alone just doesn’t cut it. To be effective, testers need to be involved in all stages of the project lifecycle (aka. testing throughout the lifecycle).

Different case studies have found that a bug found after release can cost 500 to 1500 times more to fix than if that same bug had been found at requirements. Obviously this implies that the bug was introduced during requirements, but this isn’t always the case and is beside the point. What if someone had been there to challenge what everyone assumed that particular requirement meant? What if they asked the tough and often times, obvious questions? Normally this is the job of the requirements analyst(s) on the project, but if you work on a smaller team like mine, there isn’t one. In this case, it must become the tester’s responsibility.

During Requirements
Requirements lay the foundation for everything the software is to become. If a foundation is weak, what happens to the building placed on it? Right, it either fails to be built at all, quickly crumbles to the ground, or costs tons of money to fix. A tester’s goal during requirements is to ask questions to get everyone on the same page. If two developers each make an assumption about a requirement, they could each make a different assumption. Worse yet, what if both of their assumptions are incorrect? Now we have two pieces of the system to fix instead of just one, not to mention that those pieces will likely be incompatible with each other to begin with (Continuous Integration can help with this problem, among other things).

An Example: We have a requirement which states: “The house shall have a front door.”

Is this requirement ambiguous? You bet it is! This is an easy one to make an assumption about, but what is the correct solution? Consider the following (yeah, yeah I know the pictures suck):


Each of these solutions is valid by the above requirement, but which is the correct solution? Even this one could be more correct, depending on the customer’s intentions:


The point is that based on the above requirement, all these solutions are valid. At this point, the tester can ask his/her self, how would I verify this? Just check if it has a door on the front of the house? But Where?

From a testing perspective, all requirements should be verifiable and quantifiable.

Who is qualified to answer this question? The architect? The homeowner? The contractor? The painter? Chances are that in this case the homeowner would have the final say (they’re paying for it aren’t they?), but what if they’re building the house for someone else? The requirement should be refined by the primary actor/actors in the use case(s) mapped to this requirement. In this case, it’s probably safe to say that the homeowner is the best choice (unless they really are building the house for someone else, in which case we would go to them). So, we contact the stakeholder and ask them to be more specific, and we get the following refined requirement.

The house shall have a front door on the first level.

Which is the first level?

The house shall have a front door on the ground floor of the building.

Where on the ground floor?

The house shall have a front door on the ground floor of the building, in between the two front windows.

And we have a winner!

This requirement appeared obvious at first (as so many do), but upon further inspection couldn’t have been much worse. As testers, requirements are our most precious artifact. They give us the means to verify everything that comes after it. In the absence of a dedicated requirements team or requirements analyst dedicated to our team, we need to make sure these requirements allow us to do our jobs. Ideally the developers shouldn’t have to guess even once during their implementation, but when they inevitably do they should know who to get in contact with to have their questions answered.

Copyright © 2017 Life, Teams, and Software Engineering

Theme by Anders NorenUp ↑