Skip to main content

What role are your tests serving.

We had a discussion at work after watching the excellent Kelvin Henney talk about good unit tests. And it helped me clarify the many roles tests play and why people get a habit of complaining about their tests getting in the way. So here is the four categories a test is trying to balance:


  • Requirements
  • Regression
  • Documentation
  • Refactoring guide


Most test today are written as requirements. And this is a huge improvement over the past when we'd write tests later and try and get code coverage. Those 'later' tests are useless. The requirement tests happen for high level acceptance tests and in low level TDD too. Your test defines a behaviour you want. Once your test passes you don't go back to change it at all beyond some obvious cleanups like code duplication. These tests allow rapid development and reliable progress. But they can hinder refactoring, provide poor feedback on failures for regression and may be too implementation focused to be readable.

And easy correction is to make tests act as good documentation. Unit tests should explain component behaviour to other programmers. Think of it as the examples section in your book that any programmer is going to flip to and read first. Higher level tests can use BDD, tables or other techniques to create highly readable descriptions that can be shared with your product owner or customer. One thing you'll notice if you try and make unit tests act as good documentation is you'll want to avoid the london-style or mockist style test where every secondary object is mocked out. All those mocks heavily hinder the tests being a document as it shifts away from being an example of how the code runs on the product.

You may think all those requirements focused tests are all you need to cover the regression case. But for good regression tests you want highly informative error messages. I find those get introduced later when you start seeing such failures. That might be when a bug is introduced or a new feature makes old tests inaccurate. That's a good time to go back through those assert statements and make sure they give enough information to know how to correct it. As once corrected those asserts will benefit you ever after. There are also some frameworks that can help like the highly informative asserts from py.test.

Unit tests are your best guide for refactoring. So you want a complete suite of tests that run very fast. That way you can change a line of code and know within a few seconds if that change is okay. Higher level tests are helpful too in that you could replace an entire component (perhaps something custom with something off-the-shelf) and see everything still works. Can you replace your database with confidence? That's the kind of feedback high level tests can give to help refactoring.

So a single test can satisfy all four of these needs. But it doesn't need to so it's better to figure out where the focus is for the test you are writing at the time. Some tests might only be there for documentation. It's okay for tests to evolve to help refactoring only when you need that refactoring. But don't assume that once a test is passing that it automatically ticks all these boxes. It's just a check at that point.

Comments

Popular posts from this blog

RestFixture

So most of the tests I'm writing now in Fitnesse are using RestFixture . Being able to do all this black box style testing has helped me get a lot of tests up and running without having to change the existing code base. Now I've taken a step future with my own little fork  so I can use scenarios and build nice BDD style scripts. But first I want to give me own quick guide to using RestFixture Step 1: Installing You can dive straight in by grabbing the latest jar files for RestFixture here  https://github.com/smartrics/RestFixture/downloads If you know what you're doing can get the nodep version to work nicely along side other libraries you may be including in Fitnesse. But I grabbed the 'full' version and unzipped it into a RestFixture folder alongside my FitNesseRoot folder. Step 2: Write your first test I took advantage of the built in Fitnesse api as a basic test and wrote a page called RestFixture with the following contents !define TEST_SYSTEM {slim} !

A brief introduction

And then I'll get into my first rant. I'm some guy who thinks about stuff. Stuff you probably don't care about but I'm going to tell you anyway! Now I don't think I'm going to cure cancer or create world peace but it'd be nice if I could make the world a little better. Stuff I'd like to talk about will focus on: Games I've always been a game player and am fascinated in how one designs a game. Yes it's mostly computer games these days and even then it's mostly consumed by World of Warcraft . Shut-up you in the back! We all have our vices and I'll avoid sniggering at your hentai collection if you leave my level 80 mage alone. Programming Only 2 years ago I was introduced to this weird concept of Test Driven Developmen t and the broader concept of Agile . Programming has been part of my life since I was eight. When I haven't been doing it professionally I've found myself doing it more in my free time. So it's nice that after 20+