How much does it cost to have a defect in your product? I’m sure you’ve wondered this and possibly even had conversations around it. I know I have certainly had many conversations with my technical teams as well as with my peers on various leadership teams around this very topic. Its not a question of whether there will be bugs, rather it’s a question of how many and what it costs to fix them. Let’s take a quick look at what a single defect might cost the company in terms of real dollars even when its found fairly early in the process.
A simple example
What does it cost the business when your software development process doesn’t catch a bug early in the process? Assume you have a developer working on a new piece of functionality, happily drinking caffeine and writing code at a cost of $80/hour. That developer completes the feature and hands it off to QA for testing. Let’s further assume that QA person costs the company $60/hour. Finally, let’s semi decide that it initially took 40 hours to implement the new functionality and then 16 hours to test it. The implementation cost of the feature would therefore be approximately $4,100.
That $4,100 is, obviously, the ideal cost of delivering the feature purely from an implementation/testing point of view. Now, let’s assume the QA person finds a bug that stops them from completing their testing effort. The QA person must first figure out how to replicate the issue, they must document the issue and then undoubtedly also spend some time discussing the issue with the developer. Additionally there will be some time spent by a broader team to review the issue to determine the severity of the issue and decide whether it needs to be fixed now or if it can wait until later. By making a few assumptions around how long that all takes and who gets involved its easy to show that a single bug costs 5-10% of what it cost to implement the feature in the first place. In our little example, that’s $200-$400 for this one single bug.
But wait
How many projects have you seen where one developer implements one feature, that feature is tested by one quality assurance person and that person finds only one bug? Happens all the time, right? I think not.
Let’s consider a small team of 5 people (3 engineers and 2 QA) in an Agile/Scrum process running 2 week sprints. Assume that over the course of a 2 week sprint, the team finds and fixes 10 bugs in newly created features. Those 10 bugs raise the cost of the sprint in staff payroll alone from approximately $17,000 to over $21,000. That’s almost a 30% increase in a single 2 week period. Now, apply that to a longer project over a number of months you can see how those costs would quickly add up to something non-trivial. Those same 5 people with that same 10 bug/sprint pace would increase the overall staff cost alone for a 6 month project by a $57,000! Do that for 2 projects and you’ve spent the equivalent of another full-time developer for a full year.
And that’s still not all of it
Everything up to this point has been presented in very tangible numbers but there’s a lot more to it than that. Consider the cost to the team when they are forced to task swap between new feature implementation and bug fixing. What does it cost if you factor documentation changes into the mix? Worse yet, consider how much the tangible and intangible costs go up when a defect goes undetected to the point of making into production or is found by the actual customer. Clearly the example above is only the tip of the iceberg.
You’ll obviously never eliminate every bug from the process but I think its clear that the more that’s done to catch defects bugs early the lower your development costs will go. The lower your development costs the more you can do in terms of new development.
Sure but, how?
That’s all well and good you might say but, where to start?
Tightening up the implementation process is the first and most obvious place to start. If your dev team isn’t doing code reviews (or are doing them but not well), start there. If you aren’t doing continuous integration builds, you can gain a lot there. There are tools available to help make code reviews and continuous integration much easier and more effective. Design and code reviews done right can eliminate huge numbers of bugs. Finally, engaging QA earlier in the implementation process is also a possible quick win. Get your QA people thinking about the problem before the software drops in their lap.
Consider your support process. Can an engineer be peeled off of the development cycle to be readily available to quickly solve those tough support issues? Is there a way to flag critical support issues more quickly so that the important stuff bubbles to the top?
Creating an efficient and effective development process isn’t something you do all at once. Gather data, identify problem areas and attack them one at a time. Over time, and with the right guidance, you can dramatically improve your process and lower your development costs in the process.