Economy and context behind written bug reports

Republishing my article here from an old blog site. Desk checking is still, to date, one of the most useful and widely adopted practice of the agile teams I have coached; here’s a breakdown of how it works.

A busy Friday afternoon and we’re holding the regular bi-monthly bug review. The project manager has prepared a list of issues to discuss and displays it on the 52-inch conference room TV. The product owner, business analyst, test lead, and application support analysts gather around and discussed each ticket one by one.

There were 100+ bugs in the backlog and ~50 of those are new. The review started smoothly — each issue is assigned to a release branch or reassigned back to the reporter to provide more details. After the 10th, each report became more painful and I can feel the fatigue and discomfort from each of the attendees trying to understand the report.

WHAT’S THE PROBLEM HERE?

Bug reports is one of the most important medium for anyone to communicate information about the quality of the product. Ongoing efforts everywhere are being made to make it efficient (e.g. quick logging) and effective (e.g. containing the right info), however some pursuits fall short of overseeing the context where the bug report exists.

Some bug reporting anti-patterns I’ve seen are:

• Long Summary. Some of the reports just had too much in the summary e.g. “[Mobile] [Home Page] [Scores Widget] Scores are not updated in real time unless you refresh the page”
• Vague Summary. You won’t be able to guess what the problem is at first read e.g. “Error appears in Profile Page”
• Confusing Screenshot. A series of screenshots to (hopefully) tell a story; a screenshot without annotation; a screenshot that captures only the problem element. Indeed a picture paints a thousand words, but it needs context to paint the right message.
• Too Much Information. In an effort to cover everything, the report contains pre-conditions, details steps, post-conditions, expected results, actual results, reference to requirement, browser details, etc.

I don’t intend to say these practices are bad; only that it could be unnecessarily costly.

While the pursuit of efficiency and effectiveness are good endeavors, it needs to be holistic and wary of:

• Logging Cost. Time the reporter spends browsing through the report template, writing the report details, and attaching files and screenshot among others; for software-based logging tools, costs also include disk space, server bandwidth
• Reviewing Cost. Time and energy the reviewers (application support, testers, developers, analysts, product owners, other stakeholders) need to spend reviewing each of them; brainpower also comes into the mix
• Maintenance Cost. Time, energy, and capacity spent to update, comment on, compare with other bugs, compare against current priorities, re-rank, and close each report
• Emotional Cost. Yes, there is this too; the reporter’s perception of a report template; reviewer’s patience before, during, and after reading (and re-reading); developer’s or project manager’s stress level before, during, and after reviewing — you get the idea.

WELL, THERE ARE WORKAROUNDS

Humans by nature tend to do workarounds to make their work easier or to escape difficult situations. You can see the same during bug reviews where teams make snap judgment on bug reports, prioritizing them without understanding the full context, throwing them back to the reporter for more information (which btw also results to delays), or worse, dismissing them as irrelevant.

This is a consistent pattern of behavior. I’m sure you’ve also seen managers responding, “well they need to do this correctly” (i.e. team needs to review them diligently or else…, the testers need to be providing enough information or else…).

Except, “correctly” isn’t as intuitive.

SO WHAT CAN WE DO?

Before you even think about how to write reports well, revisit first the entire value chain of your reports. Make sure you factor in the costs discussed earlier while doing so.

• Are all reports need to be logged?
• Are all reports need to be reviewed?
• Should it be reviewed by everyone at the same time?
• Are there quick decision models or heuristics we can agree on to decide the fate of a bug?

Also take the time to dig into each phase of the bug’s value chain and decide how you can do things differently:

• Collaborate and Fix ASAP. If you and your team can agree to fix them right away, then do so; it removes the need for logging entirely. Most testers are adamant of logging everything so they can prove their work, but that’s sub-optimizing a process (testing) at its best at the expense of delivery time, collaboration, and team accountability.
• “Soft-log” them. If you have discussed the problem with your developers, it may be simpler to write a post-it and stick it to their monitors or put comments onto a task stub than create an entire report. This only works if you can fix them within your iteration, else if it needs prioritizing then it needs to be logged in the tracking system.
• Focus on the customer when you write them. If you have to write a full report into a tracking system, then write them keeping your stakeholders in mind.

MINIMIZE OUTPUT, MAXIMIZE OUTCOMES

Bug reporting is an easy target when we talk about ‘minimizing waste’. Unless extensively documenting them add value to your business, seek to either fix bugs you find right away or track them minimally so it does not interfere with more important work.