Test cases are awesome

One of my major background projects for the past few months has been to extract common libraries from various codebases, and get everything into a global, unified tree. As an example of this, I have a set of string utility routines I've been using for many years, and my normal way of using them was to grab and copy the code into various projects as I needed it. For routines as well-defined as these, it wasn't really a problem, but for anything even remotely more complex, it becomes a huge maintenance nightmare.

As part of this consolidation, I now have around 20 standalone library parts I can pretty easily use. There's a unified notify/error reporting mechanism, but most importantly for the purposes of this post, there are test cases. Lots of test cases.

I have traditionally done very little in the way of formalized testing, but by the same token, I've only recently started switching to a true 'library' structure for my code. It turns out that test cases are way more powerful and useful than I ever gave credit to.

So how do these test cases work? First, the build system compiles all the libraries, then it automatically builds and runs all the test cases for them. Every time. Running all the test cases currently adds about four seconds to the build time, which I can live with. I also make sure to update the test cases as I find bugs in general usage - it's very hard to guess all the bugs that might happen in advance, so putting in new tests as bugs are discovered is pretty important.

Part of the reason this is so powerful is that it's changed my debugging method for most of the library code. I get a questionable result in the mainline, so instead of trying to debug the mainline, I go add a test case and fix the library before I even try the mainline code again. Usually after doing that, the mainline bug goes away.

Another reason the tests are so powerful is because you never have to worry about regressions, or if something you changed accidentally breaks something else. You've also got simple, clear, and obvious code that demonstrates how to use the module in question, including ways to abuse it which produce well understood errors.

The downside to doing this is time and effort. A lot of time and effort. If I had to hazard a guess, I'd say that building proper libraries and the associated test cases takes 3-4 times as much work as just hacking something together that functions. The key to making libraries pay off is to make up for the difference one of two areas: long term maintenance, and use in multiple projects.

If you expect to maintain a project for many years, you -might- make up the difference in maintenance costs.

If you expect to use the library in at least 3 projects, you will make up the difference in implementation time.

Another downside is that building libraries and test cases is decidedly unglamorous. There's nothing cool about it, nothing you can really show anybody, and if you do try to show someone, you'll get a response like "it took you 12 hours to code up a way to do dns lookup from inside the game server?" The thing that will be missing is that along the way, you added a platform independent threading and mutex model as well as an async DNS caching system. Users will see those words and think 'blah blah', and wonder why your time wasn't spent doing something productive.

Anyway, I'm sold. I'll be adding a lot more test cases in the future.