can a project be too small for test automation?

eBooks by

Can a Project Be Too Small

for Test Automation?

by Dustin Rothwell

www.intertech.com + 800.866.9884 @intertech.com 1

Table of Contents

1. Introduction .................................................................................. 2

2. The Scenario ................................................................................ 4

3. The Result .................................................................................... 7

4. The Verdict ................................................................................. 13


Introduction


Note: Throughout this article, I refer to “test automation” or “automated testing”. This includes unit tests, but is not limited to tests that are strictly classified as unit tests. This includes all developer written automated tests, whether they be unit tests, integration tests, acceptance tests, or other.

I was asked this question some time ago, at the start of a new project. Experience has taught me that automated tests are essential for developing and maintaining quality software. However, this particular project raised a question to which I had not given much thought: can a project be so small that writing automated tests is not worth the effort?

The Scenario



The project in question provided a tempting example. It was a “one- time only” application, meaning that it was intended to be run in production only once. It was also small in scope.

Based on the requirements, I estimated that it would take just a couple of weeks to write the code. With a code base this small, I thought it wouldn’t be any trouble to write a program that meets the requirements, even if I skipped writing automated tests. Plus, maintainability wasn’t an issue, since the program would only run once in production.


Therefore, I decided that this project did not need test automation. After all, it does take time to write tests. This time investment is typically justified by saying that the tests will save time in the long run. I completely agree with that, but my thinking in this case was that with such a small project, perhaps the time spent writing tests would not be justified.


The Result


If you have any experience in software development, you know that things change: projects change, priorities change, requirements change.

This case was no different.Initially, things progressed as expected: coding and (manual) testing took just a few weeks. Then the requirements changed. No problem. I updated the code and re-ran my manual tests. Then the requirements changed again. Once more, I updated the code and re-ran my manual tests.


The requirements continued to change. I continued to make coding changes and re-run my tests. This was getting annoying, but I was able to keep up with the changing requirements. Since this was a simple batch program, it was fairly easy to run manual tests by providing input files with test data, and executing database queries to initialize and reset my database. Even so, the amount of time that I was spending running these tests was adding up.

The project was de-prioritized and sat on the shelf for a few months. Then it was re- prioritized, with updated requirements (surprise!). I opened up the code and soon realized that I didn’t remember what most of it was supposed to do. Instead of having several small classes that focused on doing one thing well, I had a handful of large classes that tried to do too much, and were difficult to understand. I have enough experience in the world of software development to know the value of well designed classes, but without unit testing it was too easy to fall into bad habits.


I updated the code again to match the requirements. At this point, I was finally starting to think about refactoring the code to make it cleaner and easier to maintain. But, I thought that surely the end must be near and this project can finally be put to rest, so I continued down the path of making the code uglier and uglier.

Additionally, refactoring would have been very difficult at this point. After a few months of not working on this project, I had forgotten how to properly test it. Have you ever tried to refactor a code base that had no test automation? It is difficult to make even a small change and know whether or not you just broke something. What a difference this was compared to other recent projects that had good test coverage. You could make a change, run the tests, and be comforted by all the green lights. Without that security, every little change is accompanied by anxiety.


I am not saying that test automation guarantees your code is easy to maintain and bug-free. That largely depends on the quality of your tests. What is true, in my experience, is that over time you will gain a certain confidence level in your tests.

You will come to know how accurate they are; how likely they are to catch defects. With good tests, this confidence will become high. But, even with tests that are not so great, you will at least have some level of confidence.


In this case, the downward spiral continued: the project is put on hold, changes are made to the requirements, code gets uglier. This went on for several months.

The result was a code base that, while still very small, was difficult to maintain. It was hard to change one thing without breaking something else. Every time I opened the code to make a change, I had to hang my head and cry at the mess I had created. I considered stripping my name from the code as an attempt to hide my shame, only to realize that it would be of no use. Thanks to source control, my name was forever linked to this festering abomination.


The Verdict


Back to the question: can a project be too small to justify automated testing? In theory, the answer is probably yes.

If you have a very small project where the requirements never change and everything goes according to plan, then perhaps it is not worth your time to write tests. In reality, however, this type of project does not exist. At least, not in my experience. Business needs change. Projects change. Requirements change.


Test automation helps you deal with these changes. Instead of being paranoid about implementing a feature, or refactoring a certain piece of code, you have confidence that regressions will be caught by your tests. Without the benefit of these tests, it is easy for things to spiral out of control. The code becomes more and more unwieldy because you are afraid of refactoring, for fear of breaking something unexpected. The end result is a steaming mess of a code base; one that will have your name on it.

After this debacle, I resolved to never treat a project as being “too small” for test automation. Even if it seems so at the beginning, chances are it won’t stay that way. And, as I learned from this experience, the cost of not writing automated tests becomes very high, very quickly.


Need help with a development project? Speak with a Consulting Representative today.

651.288.7001

See our list of Consulting Solutions. Consulting Solutions

https://www.intertech.com/Consulting

https://www.intertech.com/

https://www.intertech.com/Consulting

can a project be too small for test automation?

Software