We crank out bug reports and expect them to return like a boomerang so we can check to see if the bugs were fixed. In this week's column, Danny Faught shares some ideas drawn from recent experiences that could make you a better customer advocate subsequent to filing a bug report.
Before the Bug Returns
The people on your team may add comments to a bug report as part of an ongoing dialog about how to handle the bug. This gives you a nice history about decisions and additional details about the nature of the bug and the fix. After you file a bug report and send it on its way, there may be more opportunities for you provide additional useful information.
On my current project, the programmers tend to add comments to the bug reports assigned to them when they learn new information. They also may add comments and forward the bugs to the development manager, seeking feedback. These are opportunities for me to see if there's more information I can provide about the nature of the problem that I didn't think to include when I first filed the bug. Also, I may comment on how well I think any proposed fixes will work. That greatly shortens the communication path compared to having the programmer completely implement a fix before I have a chance to review it.
Our bug tracking system notifies the team when a new comment is posted, so it's easy to track these conversations. If your system makes it more difficult to see comments posted, or if the volume of activity would make it impractical to monitor the things you're interested in, you might have to wait until the boomerang comes back to you. Also, if the programmers are hostile to receiving feedback when they're working on a bug, you're better off following a more formal process.
When the bug is fixed, more or less.
The boomerang returns...In many organizations, the people who file bug reports are asked to verify the fix when the fix is completed. Or you may be testing the fix for a bug someone else submitted. My goal is to complete this step with at least as many bugs open as when I started.
Of course, you should try to reproduce the bug the way you originally reported it. Sometimes, the problem is still there, as if nothing at all was fixed. Perhaps because of a configuration management glitch, the fix isn't in the build you have. Or maybe there's something specific about my configuration that I either didn't report or the programmer didn't think was significant. When I reject a bug fix outright, I try to include additional details about my configuration and exactly how I reproduced the bug. I use different wording than I did the first time around to help reduce any misunderstanding. And I write with a tone that expresses puzzlement rather than condemnation. In almost all cases, the programmer really did make an effort to fix the bug, and we need to recognize that effort.
What if the fix did some good but doesn't address the bug to your satisfaction? You have a judgement call to make. If the changes to the software represent some sort of progress, and can stand alone if no further improvements are made, then I recommend closing the bug as fixed. I don't like keeping a bug report open for a long time, spawning many changes, and possibly changing its focus several times.
Open one or more new bug reports, describing the additional changes you would like to see. Doing this before closing the existing bug will help make sure you don't get distracted and forget to follow up, and it will give you the ID numbers for the new bugs that you can reference in the comments when you close out the old report. It's good to leave a trail between related bugs.
On the other hand, if the fix is really unfinished, go ahead and reject it; send the bug back to the programmer. Maybe an obvious new bug was introduced, or some small detail was overlooked. It would be easier for the programmer to clean things up while the code is still fresh on her mind than to wait for a new bug report to filter through the system. Deciding whether to reject a fix or file a new bug report is often a tough one, but if you have developed a good working relationship with the programmers, you'll get a good result either way.
While you're at it...
While I'm checking a bug fix, I sniff around for other bugs in the same area of the application. If the code is growing or changing frequently, I can often dig up additional problems to report without much extra work. Bugs tend to cluster together, and bug fixes tend to unmask latent bugs you couldn't see before.
Even if you are under strong schedule pressure, it's worthwhile to spend a few minutes to get a few more bugs filed. Organizations that support exploratory testing are more likely to see the value of this additional effort.
Not a bug?
Ouch. It's not fun when someone says the problem you so carefully isolated and reported isn't really a problem after all. Here are a few recent cases I've been involved with:
The software I'm testing has a key component that is provided by a third party. Several times I've filed bugs that flew right back to me, with a comment that not much can be done about it. This is the point where my hackles as a customer advocate go up. Customers won't care whether the problem is in our code or someone else's. They only care that the software doesn't work. In these cases, I'll do my best to explain the impact of the problem to the customer and maybe suggest a way we can work around the problem. I'll be sensitive to the fact that a workaround might be expensive to implement.
Another example is when the programmer decides that something I think is a problem is not a problem at all. I reported a bug that could cause the user to be unable to log into the application for several hours. The developer said that this was the expected behavior, and the development manager agreed. I was out voted, but far from giving up. Rather than continuing the discussion in the bug report's comments, I sent an email to the programmer and the manager explaining why I thought it was a mistake not to fix the problem. Not only did they agree with me, but the manager also validated my opinion by posting my message into the bug's comments. But the saga didn't end there. I thought that the fix that was implemented didn't go far enough to address the problem. Since the fix did work as designed and didn't introduce any new problems, I closed it out and opened a new bug report asking for more. Given the improvement that was made, we were all able to agree that this new request is valid but a lower priority.
The key to being a good customer advocate is to consider the human elements of the development process. Practice the fine art of holding your ground on issues that affect customers while maintaining effective communication with the programmers and managers. Hopefully I've given you a few useful tips for doing that better.
