ICS 22 / CSE 22 Fall 2012, Unit Testing and Test-Driven Development with JUnit

Introduction

I asked you fairly early in the demonstration not to take notes, because I wanted you to be able to concentrate on the process of test-driven development by following the example step-by-step. This document, combined with the step-by-step code examples, are meant to take the place of the notes you might have taken. I've also taken the liberty of saying a few additional things that I didn't have a chance to talk about in lecture.

The process

Test-driven development encourages you to build a program one small feature at a time, taking small steps from one piece of stable ground to another. The notion of "small feature" is open to debate, though a good guideline is to prefer features as simple as "The size of a newly-created collection of songs is zero" over features as complex as "A complete SongCollection class." The goal is to write a test that verifies the behavior of the new feature, then to write the code that implements the feature, using the test as a guide to indicate when you're done. At this point, you'll have a feature that is complete and tested, which means you've taken a step on to stable ground; more importantly, you have a test that you can keep forever, which you'll be able to run repeatedly to ensure that your feature still works as you make changes and add new features to your program. (Contrast this approach to the one you've taken as you've worked on your programs thus far this quarter. With your current approach, how do you know that some part of your program is finished? How do you ensure that it continues to work correctly as you continue to make changes to your program?)

In lecture, we went through a step-by-step example as a group, developing portions of a SongCollection class using a test-driven development process. We did our best to follow all of the steps, though we sometimes forgot (or took liberties in the interest of time). Because it's so different from the programming style we're accustomed to, it takes a little time to adjust and get into the rhythm of test-driven development. But don't let the learning curve chase you away! It doesn't take long to get adjusted, and the benefits are higher-quality code — in terms of both how well it works and how well it's designed — and the ability to make changes to your program with confidence.

Pick a new feature that you want to implement, preferring very simple features that can be verified with a single test. (It's not that you can't implement complex programs using test-driven development; it's just that you have to break them into simpler pieces. This is a good practice whether you're using a test-driven philosophy or not.)
Write a test. The test is intended to verify the behavior of a feature of a class (or classes) that very likely hasn't been added yet, which means you'll potentially be creating objects of classes that don't yet exist, or calling methods that you haven't written. This may seem weird, but it's actually the whole point; pretend like the classes and methods you want have already been written. There are at least a couple of benefits to writing the test first:
- You won't need to guess whether your code works; the test will tell you when you've successfully implemented the feature.
- You've tested your design before you've ever implemented it. If you discover that the code that sets up the necessary objects and calls the method(s) you're testing seems more cumbersome than it needs to be, that is a very good indication that your design is probably more convoluted than it needs to be. Your design is at least as important as the code you write; a clean design ensures that your program will be understandable (to the original author and to others), as well as being maintainable and extensible as users request bug fixes and new features. These qualities should not be underestimated; programs in the "real world" often live a good deal longer than the original authors intend (and often stay in an organization long after the original author has moved on to greener pastures), and it's important to be able to introduce changes to a program without it falling down like a house of cards.
It's wise to start with very simple features and work your way up to the somewhat more complex ones, which is why we chose to begin by testing that the size of an empty collection of songs is zero.
Compile the test, even though you know it won't compile without errors. The point here is to get the compiler to tell you what you're missing, rather than guessing at it. After compiling the test and reading the error messages, you'll have a clear idea of what code needs to be added (or rewritten) in order to make the test compile successfully. (As we saw in the demonstration, Eclipse automates this step, because it recompiles your code every time you save it.)
Write the minimum amount of code in your class that will make the test compile, without worrying about whether the class will then pass the test. The objective here is to declare the new methods of your class that will be needed by your test, but not to implement them yet. We'll let the test tell us what code is missing in the next step.
Run the test. It will almost always fail, and when it does, it will tell you precisely why, by telling you what behavior was expected and what behavior was observed. Occasionally the test won't fail the first time, because the "stub" you wrote is (accidentally) correct. For example, we initially wrote this method in the SongCollection class when we tried to get the "Is the size of an empty collection zero?" test to compile.
```
    public int size()
    {
        return 0;
    }
```
It just so happened that returning 0 was the right thing to do, though the reason we said "return 0" was because, in Java, you have to return some integer from the method in order for it to compile. If the only goal is to get the method to compile, saying "return 0" is as good as anything else. Luckily, 0 was the right result; it usually won't be.
Assuming that the test failed (as it usually will), the test has told you specifically why it failed, so you should write the minimum amount of code in your class that will make the test pass. This is a difficult habit to get yourself into at first, because it often necessitates writing code that works perfectly in the simple case you're testing, but clearly won't work later on. That's okay; you'll be able to write code for the more general case later, and will have all your old tests so that you can verify that the simpler cases, as well as all the other functionality you've already built, still work correctly after the change. The tests are not something you write and then throw away; you'll keep them for as long as you keep your program, so that any time you want to go back and make changes anywhere in the program, your tests will be available to verify that nothing else has been broken as a result.
Run the test again. Hopefully, it will pass, which means that your new feature is implemented! You've now reached stable ground. (With the approach you've been using so far this quarter, how often do you feel like you're on stable ground?)
Now that you have your new feature implemented, see whether there are any ways to improve the design of the code. (We're looking for what are often called "code smells": places where the design could be made better.) Have you duplicated code from another part of the class (or from some other class)? Did the code you just added render older code useless? If so, fix the problems now, running the tests after each small change. (There's a name for this process; it's called refactoring.) You can make changes with confidence, because your tests provide a valuable safety net; if some change you've made breaks code that once worked, your tests will tell you so immediately, so you can work on the new problem while the change you just made is still fresh in your mind.
Now start this process again with another feature. Continue this until you believe that all of the features of your program are implemented.

After going through one iteration of this process, you'll have added one new feature to your program, verified that the feature works as expected, and cleaned up any brewing design problems before they become significantly bigger problems later. As the JUnit folks say, "Keep the bar green to keep the code clean."

What if I still discover a bug?

We didn't talk in lecture about what should be done if you discover a bug in your program, even if you've faithfully adhered to a test-driven strategy. Naturally, using a test-driven development process does not guarantee that a program will work, for a variety of reasons. Following this process allows the compiler and testing framework to help you avoid many mistakes, but there are many other aspects of software development that this process doesn't do much to improve. First of all, your program only works as well as your tests say it will; if one of your tests expects behavior that is incorrect (e.g., the size of an empty collection is 1) and you write code that passes the test, that doesn't mean that the code makes sense in a broader context. Similarly, tests can't verify that the program's requirements are appropriate; if you are tasked with building software that won't meet the business needs of your customer, tests won't help you identify the issue. In short, testing helps verify that a program is correct, but the notion of "correct" often isn't a black-and-white one.

So, unfortunately, there will still be bugs. The question is what should be done when you discover one. The following steps can guide you through your bug-fixing:

Write a test that reproduces the bug and asserts that the unintended behavior shouldn't happen. This step is critical, because it will provide you with a way of being sure that you've actually fixed the bug.
Run the test to verify that it fails because of the bug. If it doesn't, you haven't isolated the problem, so you'll need to go back and write a better test.
Find and fix the bug as you would normally. (If you find that you need to add new features to your program in order to fix the bug, follow the set of steps described above for adding them carefully, writing tests first, writing the minimum amounts of code needed to make them work, and so on.)
Run all of the tests to verify that the bug is fixed and that all of the other tests still pass, as well.

Now you can have confidence that you've not only fixed the problem, but also haven't broken anything else that previously worked. You'll again reach stable ground quickly.

Additional thoughts

Give this process a genuine try, even if it feels less productive — or just plain strange — when compared to your usual strategy for writing your programs. Trust me; if you can get yourself into a rhythm, you will find yourself writing higher-quality code more quickly, with less mistakes early on and less debugging to do at the end. As we learned from our experience in lecture, test-driven development works very nicely with pair programming. I sometimes made mistakes in my haste to get code written while still explaining everything to you, but with you folks working collectively as my "partner," we ended up with virtually no mistakes that lasted longer than a few seconds.

Above all, have fun! Developing software should be an exciting, enjoyable, and stimulating experience. Test-driven development takes away a good deal of the frustration involved, allowing you to concentrate on understanding the problem and constructing a clean solution for it.