Most successful products eventually move to a maintenance phase, during which the team has to balance between adding more functionality, keeping up with the timeline and maintaining a sufficient level of quality. One of the questions many small teams encounter is what they should do with all those minor bugs found in the product, especially given budget limitations. We will try to help your team define your own strategy of getting rid of non-critical defects in your bug tracker which would suite you the best.
Part 1. Understanding the problem
Ben had a stressful morning today. His customer called, complaining about a few issues they saw, so Ben had to schedule another hotpatch and re-assign resources, taking them from a new feature which had to go out in the next release. That meant that a few people who worked hard to stay on schedule and accomplish their primary goals had to switch context, work on a hotfix and follow up with the customer on the fix. The day became more stressful when he realized that the issues reported by the customer were known at least 4 releases ago and were already piled up with other minor bugs on a dumping ground they called “bug tracker.” He opened the issue list. 2,000 open issues that nobody wants to touch. How did it become that messy? It’s not like they were not doing bug fixes at all. During each release, they were fixing all the major bugs and left the known minor ones for later. But that later never came.
What is the source of evil in this story? A few things, primarily the absence of an accurate strategy for dealing with minor issues, and no balance between scope, time and technical debt.
Assuming you are working on a long term project, you will probably find yourself in a situation when your bug tracker becomes a collector’s edition for all sorts of minor bugs. How do you find your way out? What are the possible avenues?
The following is not a step-by-step instruction - the contexts of different projects are too different. Instead, the following should help you define your own strategy for dealing with bugs.
Where to start?
The first thing you have to do is to understand the problem. Allocate some time to ask yourself the following questions:
- Have you ever found yourself in a situation similar to what Ben faced?
- Have you ever felt desperate when looking at your bug tracker or dashboard?
- Have you ever procrastinated when you should have reviewed open issues?
- Have you ever delayed your release date due to a large number of last-minute bugfixes?
When the problem is defined, find the bottleneck in your defect management process. Here are some leading questions:
- Which stage of your process adds to the pile growth or release delay?
- Does your team file appropriate bugs?
- Are issues communicated, reviewed and prioritized accordingly?
- Do you take actions on all issues filed and do you have enough information to take action?
- What does the bug life cycle look like in your team?
Identify the cause. In a retrospective, try to remember the context and actions (or no action) that led to the bottleneck appearing.
Finally, take action. We need to solve 2 problems now. First, come up with a plan on how to deal with the 2,000 issues in your tracker. Then, create a working process of preventing the same situation from happening again.
Defining a strategy for taking action
There is no silver bullet that works in every case. But here are general recommendations that worked at our company which you may apply.
General approach
Your plan will be the following:
- Communicate the problem
- Define and apply a short-term solution for dealing with existing issues
- Define a preventive process for the same case:
- Select the strategy
- Take action
- Check results
- Keep improving the process
So, first of all, don’t hesitate to communicate the problem with your team. They should be aware of the benefits you are aiming to gain by solving the problem. That should prevent resistance for change. They also might suggest brilliant ideas for solving the problem.
Part 2. Cleaning the Augean stable
Now, let’s start with dealing with that pile of bugs.
First, let’s imagine your bug report review workflow for every single issue. If you want, you can take a pencil and visualize your workflow as a flowchart. Now, for each box, put some time estimate - whatever it takes on average in your day-to-day basis. Let’s look at this workflow as an example:
Now, multiply the given numbers for time estimates for time estimates by 2,000.
Though the estimate is really raw, now you can imagine the required effort to review the issue list once. But that’s not a one time activity - you won’t be able to fix all 2,000 issues in the upcoming release, so consider multiplying that number until the pile is dealt with. We also didn’t calculate any bug fixing and bugfix testing efforts.
Though it sounds scary at first glance, you might find out that just closing the old issues that had no action in some definite time would be much more cost efficient for your project than to review and fix them, if the product in its current state is of sufficient quality. Given the bugs we are reviewing were all defined as minor, and no customers complained in all those years, maybe that is true for you - but see for yourself.
That’s not necessarily a solution that would suit you. Again, the contexts and quality expectations vary from project to project. Just define what suits you best - your goal is to take action on that pile of bugs.
If you see that indeed closing old issues is the best solution in terms of efficiency, then set a date that you still consider valid for bug reports and just close issues which are older.
For the remaining not-so-old issues, schedule some time in the project timeline and categorize issues as follows:
- Harmful to the business
- Not harmful
Close all items that are not harmful to the business (you can still aggregate and submit them as an improvement suggestion so they go under your feature/improvement process). Prioritize those that are considered harmful and discuss the scope with business users.
Communicate the decision with the business team. They might be against this decision at first, but they should see the economic benefit. They also might provide your team with a list of business-harming issues they think are critical to fix.
If the project context requires it, communicate the decision with customers. That might be tricky, especially if you have an open bug tracker. You might want to invest some additional effort in reviewing your customers’ issue lists and come up with the items that should be fixed after all. But then, fix them as soon as possible and close the remaining.
Part 3. Taking preventive measures
Now it’s time to develop some preventive measures.
Let’s start with small steps. Set Time To Live (TTL) for bugs in the tracker - close them automatically when a certain time passes. Of course, you can initiate some closing activities prior to that if your process enforces that.
Decide on a percentage of minor and trivial bugs in your tracker which you can live with. Common sense dictates that you won’t be able to fix everything in a release. Set a separate value for each product area based on the risk of that area. Make sure you don’t exceed this criteria after each release, and if you do, plan your next release accordingly to meet the criteria again.
Set the definition of a defect - maybe during your review process, you will notice that some filed items are not in the scope of the project at all. Filing and reviewing such issues is a waste of project time. Communicate this definition with the team.
Do not be afraid to share responsibility. Divide the scope among your team members. You can divide the scope by different criterion. Assigning responsibility to review tasks per sub-system or project area worked for us.
Remember before how we discussed categorizing issues as harmful and not harmful to business? Add your criteria to the definition of a bug.
Set up a triage process. In short terms, the triage is a process of prioritization. You already have it in one form or the other in your team if you’ve ever set the priority for an issue. But since we divided the scope among the team, it is important to have a common vision of the prioritization process. Set a timeline for reviewing and suggest the approach, including a strategy of dealing with minor bugs. As an example, during the triage, you might want to prioritize the issues as follows:
If you think a minor issue is not critical enough to be fixed in the next 3-4 releases, and your criteria allows you to live with it, just close it. Otherwise, you’ll end up with a pile of minor bugs again.
Review the definition of a defect regularly. Involve your business team. Monitor support issues and audience feedback for better bug definitions and their severity in your project context.
Summary
Managing bugs is painful. Though setting up the process of dealing with bugs might seem like a boring and bureaucratic procedure, it is a necessary evil compared to the risk of dealing with a pile of issues, where the important ones might be lost due to a large total number of bugs or to risk delaying the project schedule when it could be avoided.
