Current Thoughts on Software Testing

When I was a younger developer I didn’t see the value in writing tests for the projects I worked on. I was focused on being as productive as possible during the day to turn out new features and fix bugs. Tests doubled or more the amount of code that needed to be written, it added delays and distractions waiting for tests to run, and sometimes they were actually very difficult to write (figuring out how to mock out things is not always easy).

That approach works for a while, but has it’s limits.

There are 3 main things that tests solve for me and it took a while to appreciate:

1. It catches stupid mistakes earlier
2. Improves ability to take on or hand off a codebase to new developers
3. Helps stave off fear of making changes in the future

Catching mistakes early is accomplished because, tests force you to think through some of the error cases, it makes sure that you’ve actually run the code before launching in front of users. I have come to see unit tests as comparable to double entry accounting in some ways – you write the credit and the debit separately, and then can do a validation step (running the tests) to confirm that both sides are in agreement. This is valuable but it is the least valuable of the 3 points that make tests worth writing. For non-critical software, and if it is quick and easy to identify and fix issues in the deployed software then it’s easy to see how fixing errors after the fact, and writing half the amount of code is a productivity trade off that has a strong argument.

Being a developer that comes onto an existing project, having tests are a life saver. If you don’t know how to quickly identify and fix bugs, then every bug becomes a lengthy investigation. This process of having new developers have to re-learn how the existing software was built is one of the biggest cost factors in software development. The time it takes to understand how previous developers decided to solve problems in a complex pile of files can be upwards of a year or more. If the average tenure of a developer on a project is only 2 years before they are promoted or move to another project, then the company gets very little time of a developer working efficiently. At this point in a projects life, having tests improves the productivity of developers. Tests allow new developers to catch issues they wouldn’t have known about, helps them learn faster about why things are done the way they are, and reduces the likelihood of introducing a bug that takes a long investigation to diagnose and fix in unfamiliar corners of the software.

As developers our job is to write software that adds new features and continues to provide value to users and that stays competitive. After a few big bugs have slipped through from lack of test coverage both developers and project managers will lose confidence in the ability to develop new things and make changes to the code. Simple library upgrades and security patches may be delayed or ignored for years out of fear of breaking things. When a team has fear of making software changes it can doom the project unless addressed. The ability to work on innovative new ideas is replaced with more mundane fixes and patching bugs. It won’t take long for things to get so out of date, so unmaintainable and so boring to work on that two things will happen: The best developers will jump ship and move onto more interesting projects and the remaining good developers will start to push for a re-write from scratch. This is a doubly bad spot to be in as a business. The parting developers take with them the knowledge and expertise that’s required to be productive leaving holes in the team that will require new people to relearn what has been lost. And secondly, the cost of a re-write can be massive and not viable, it’s risky and time-consuming. Not addressed, this will rot a project until failure.

These are the lessons I have learned from working on dozens of software projects over 15 years about the value of tests. I started as a huge skeptic. It took a while, but I’ve come around to see them as indispensable.