Friday, August 14, 2015

Can Programmers Test?

I spend a lot of time working closely with programmers on the software testing process. Much of my career mission is to get more people embracing techniques like Test Driven Development. As I push more and more of the testing demands onto the developers I sometimes run into people who say they need a tester for "independent testing".

It's a concept I'm familiar with from early days in my career. A tester can look at the system and requirements with fresh eyes and find all those cases the programmer didn't consider. If the programmer does all the testing they'll be too tempted to just test the cases they know the code is handling and not discover any new areas where bugs may be hiding. The problem I have with this idea is I never was that type of programmer.

As a programmer if I didn't test something it was purely out of laziness and arrogance. I'm a big supporter of Larry Wall's three great virtues of a programmer. A lazy programmer will do their best to eliminate bugs as early as possible to save time. But if the programmer knows someone else will spend a lot of time looking for bugs then why should he bother? Just chuck it to the test team as soon as possible for all that information. It takes either removing that test team or a higher level of discipline to break that pattern. The independent tester is not catching bugs the programmer would not have found himself. They're spending a lot of time documenting bugs the programmer couldn't be bothered finding himself.

The comparison I like to make is a writer and an editor. As a writer you need to stay focused and complete your work before you get to thinking about editing. Then you edit it. You edit it multiple times constantly revising your work until you think it's fantastic. Only at that point do you hand it over to a professional editor. I'd like to see programmers all do the same thing. You can do testing. You should do testing. You should test until you feel there are no bugs left to find. Then hand it to a tester and maybe they'll find one or two things.

Testers are not all that independent anyway. They're reading the same requirements as the programmer. They're under similar deadlines and goals to get the product out the door. Any argument that says a programmer would avoid testing something could just as equally be applied to a tester. The reality is it doesn't happen because we're professionals and understand the need for a quality product. There's no reason the demand for quality should be solely the responsibility of the tester.

What is required to get programmers more testing is education. They're not often taught about testing methods or the kind of data that makes for good tests. That alone would add great value to the tests they can quickly perform before handing over a product. But there also needs to be a shift away from the thinking that a programmers time is too valuable to perform testing. If the programmer finds the errors they can quickly fix them before moving onto their next task. The longer it takes before the bug is found the longer it takes for a fix to apply as the programmer has to open up old code, go through any required reviews again, and finally get the fix out for re-testing.

So I strongly believe programmers can and should test. But how do I get that message across to everyone?

Monday, August 3, 2015

How to Write Code Fast

Start with a clean slate with simple goals

Starting from scratch is such a rarely privilege in professional programming. We get to do it all the time when we’re learning and trying things out. In our early years getting stuck and stumbled on the most trivial challenges. And later being proud of the frameworks and architectures we can build in days (sometimes hours).
But there is rarely any money to be made in building something from scratch. There is no value in doing something trivial because it likely has already been done a thousand times before. So we need to add something to existing code. Maybe it’s something we already developed ourselves or something we bought. If we’re lucky the existing code is modular enough that the new feature can feel like starting with a clean slate and give us that burst of speed.
So you will often hear the cries from programmers that are working with that monolithic and aging code; “please, please let us rewrite this code from scratch”. I’ve made this cry a few times in my career. Sworn to make it fast enough to reduce the cost. Promising to fix all those bugs that we now endure. Brazen enough to believe there will be no new bugs introduced this time that cannot be plucked out. We lie so much. We lie to ourselves and to the company. We lie because the pain of that old code is so real and so present.
The reality is that recreating something from scratch often takes just as long as the original did. Which may be years. Is in a race to catch the original is still being patched and enhanced in use. And the switch is still horribly painful as you have spent so long writing code before it’s put into real use and only then you find a bunch more issues. So you blow the budget, often reintroduce many of the same bugs as before, sometimes a few new ones and now hate the new code so much you cry “please, please let us rewrite this code from scratch!”
So if we’re lucky we get to work on something simple from scratch. No wonder people work for startups for stock options as payment. Or if there is an existing modular system it can feel like working from scratch. So we see such frameworks are massively popular in your language of choice. But that speed may not last.

Start with high quality code with simple goals

A system that is highly modular can feel like working from scratch. It can do this by actually splitting any operation into a collection of lots of smaller applications. The unix operating system is built on this principle. No single program does much at all. But they all implement the same basic method of handling input and output. That allows us to pipe data into and out of any combination of these programs to perform increasingly more complex tasks. There is no motivation to write a new text sort program because one already exists. There is no motivation to write a new text sort function because it is so easy to call the existing program.
But additionally the code can be very well understood and structured in a way that it is easy to change or extend. When an existing code base creates a high confidence on the outcome of any change the programmers are free to change it in any way they wish. So that existing code becomes a nice launch pad to very quickly create a new variation or addition. It may not feel like working from scratch because you may be changing code rather than writing new files. But it allows small changes to be quickly released, tested and reviewed. So progress kicks off at a blistering speed.
But each change may bring with it some technical debt. Create a new link between two modules that were previously unaware of each other and you have quickly created a nice new feature. But you’ve also created a dependency that is not obvious. Now all the tests need to be aware that one affects the other. Any new changes to either module needs to be intimately aware of this relationship that got hacked in. Further features and changes create a complex web of relationships. It starts to lose that feeling of simplicity and confidence. Then the bugs creep in. The bugs you’re afraid to fix because it might break something else way over there.

Pile up technical debt (aka make a mess)

But the customer hasn’t noticed that bug yet. They just want features. It’s the new features that make money. It’s the new functionality that will fight off the competition. As long as the system holds up and functions right now there’s no time free to fight the bugs. Nevermind clean up the code and make it more modular. We want to go fast right?
I like the analogy of the application being like a restaurant. You can present a very nice front area with great service and a wonderful menu to the customers and make a lot of money. Little do they know the mess being made in the kitchen as the chefs franticly create meals. If the customers saw the mess, the pests and the rotting smell back in the kitchen they would never eat here. None them have gotten sick… yet.
You quickly get lots of features out and the customers love the product. Sure there’s a few bugs but they’ll endure it for all the value the application is giving them. But the demand for features doesn’t slow down. Each new feature is harder to do and risks introducing bugs as fragile code is pushed beyond its original function. Outside of the programming team there are murmurings. What happened to this fast pace the programmers used to manage? How did a good team become so bad?

Keep it clean and lean

So what’s left? It’s not financially beneficial to start from scratch. Even if we start from a base of quality it’s easy to lose our way and lose that base. What allows us to go fast and stay fast? Is technical debt as inevitable as entropy. Would the second law of programming read: The sum of technical debt in software increases as the code is changed and added to?
Robert C. Martin says, “The only way to go fast is to go well”. While there are plenty of other opportunities to write code fast many of them do not last. The lasting methods embrace the key practices of eXtreme Programming (XP). Pair programming, Test-Driven Development, Design Improvement, Continuous Integration and Collective Code Ownership. These are technical practices and may not mean much to customers or project managers. There are many people in a successful business that won’t be looking at the code quality. So the responsibility lies with the programmers as only they are reading and writing code each day.
Pair programming is easy to roll out but often resisted. Many programmers learned the complex world of computing as the complex world of social interaction seemed too daunting. But code reviews afterwards are a weak substitute. By then it’s far too easy to say the design is locked in now and too hard to fix. Code reviewers understanding of the solution in place is limited while pairs discover the solutions together and weigh up the options each can bring before progressing. There are many studies with good methodologies that show pair programming can dramatically reduce the number of bugs in the final product. If you care about quality the question should be; why are you not doing pair programming already? And now many are taking it even future with mob programming getting a whole team mind share.
Test-Driven Development (TDD) drives you towards a testable solution. Highly tested code is code that brings high confidence in making changes. High confidence in making changes means even late changes are easy and welcome. High confidence in making changes gives you that launch pad to make changes quickly and get results quickly. But you only get that launch pad if you were doing TDD all along. An ideal time to have started writing these tests was ten years ago. But you can settle with starting today. In the long run TDD will make you faster for a very small cost in time (10-20% according to studies). I would argue the cost is even less for complex tasks as it gets you writing code while you’re still figuring out the full solution. And now you can add to it in so many other ways with acceptance testing, behaviour driven development and more.
Design Improvement is a recognition that the system you’re building now many not be the same system you’re building in three months. The requirements will change and evolve and so too should the design. With the high knowledge of code we have thanks to pair programming and the high confidence in making changes thanks to TDD we are free to make changes that improve the overall design after the implementation. You can change the entire architecture of your code base piece by piece to eventually end up somewhere closer to the product you have now and not the one you intended six months ago.
Continuous Integration is to help recognise problems early and fix them early. If someone introduces a bug there is a big red flashing marker to stop work until it’s fixed. Much like you would expect on a factory production line when an issue introduced is noticed. The line is stopped and the problem is resolved immediately. And why stop there? You can get continuous releases going for rapid feedback from all your customers.
And finally Collective Code Ownership is critical to keeping any team of developers working fast. If your database expert is sick for a month you cannot afford to have all production slow down. If a reports system was developed by someone who since left the company how do you explain to the customer why further additions are coming so slowly? Pair programming goes a long way to resolving this but so does the TDD and design improvements. Keeping the system simple and one that the whole team intimately understands. If a developer sees a line of code they want to fix they should feel confident in fixing it, on the spot, without consulting anyone beyond the pair they are working with.
Too often Agile is rolled out without these core XP practices and while teams are releasing in regular cycles release speed is slow. Many just accept the slow buggy releases as inevitable in software development much like they did with the waterfall code of the 90s. So great is the possibilities of going fast and there is so little to lose. Asking people to put higher quality into the code and their development team should not be considered a cost. And thanks to that quality you can write code faster.


The Costs and Benefits of Pair Programming. A Cockburn & L Williams.
On the Effectiveness of Unit Test Automation at Microsoft. L Williams , G Kudrjavets & N Nagappan.
Implications of Test-Driven Development A Pilot Study. R Kaufmann & D Janzen.

Tuesday, December 4, 2012

What to test when testing Javascript

Part two in the series Test Driven Development with Javascript.

By now you've probably read and heard a lot warning you off testing through the UI and testing the user interface generally. Those still holds true for Javascript so you may be wondering if there is anything left to test. Time for some code!

Not very testable code

function validate() {
    if( == "" || == "")
        alert("Please fill out all fields before clicking submit!");
        return false;

In this there is some logic that is not directly tied to the UI. So there is something we can test. But it’ll need a bit of a rework to make it testable

function isFormValid(formElement) {
    return ( != "" && != "");

function validate() {
    if (!isFormValid(document.simple_form)) {
        alert("Please fill out all fields before clicking submit!");
        return false;

Now we have a testable function isFormValid. It’s an example of separating the logic from the UI and the DOM interaction. There is still a DOM element being used but that’s fine as it is easy to create or perhaps mock in writing the test.

Code not worth testing

$(‘.button).click(function() {
  $(‘#my_quote’).css(‘background-color’, ‘red’);

This code is still testable despite using some jquery magic. It’s easy to mock out jquery or fake events and DOM enough to test it. Do not test this code. There is no real logic in there and it’s heavily tied to the UI interaction. This may mean a lot of the Javascript you’re writing does not have tests and that’s okay. Interactions like this are still best tested manually and if they fail in a harmful way will be obvious to anyone using the system. Logic errors however can be much harder to spot through manual testing.

Testable closures

function passwordCheckClosure(password) {
  return function(comparePassword) {
    return comparePassword === password;

I love closures and closures like this are still really easy to test. Just as I would still expect you to create private methods and properties when doing TDD in C++ or Java. In Javascript I would still expect you to write closures to help build your code into clear abstract interfaces. But as with private methods and constructors you need to avoid hiding everything in closures.

Untestable closures

function validate() {
    function isFormValid(formElement) {
        return ( != "" &&
   != "");

    if (!isFormValid(document.simple_form)) {
        alert("Please fill out all fields before clicking submit!");
        return false;

This is just a slight variation on our form valid check above. But now we cannot access isFormValid to test it separate to the validate method as a whole.

 Dependency injection with a closure

// formElenment - jquery reference to the form object
// alertMethod - alert on the browser. something else for console or tests
function attachFormValidator(formElement, alertMethod) {
    function isFormValid() {
        return (formElement.find(‘#name’).val() != "" &&
            formElement.find(‘#email’).val() != "");
    formElement.submit(function(e) {
        if (!isFormValid()) {
          alert("Please fill out all fields before clicking submit!");

This may seem harder to test. We've introduced expections that we’re using jquery and put everything within a function that does not return anything. But it’s actually very testable since we can easily mock out all the objects that we send in and trigger and test those actions through those mocks. Because objects are cheap and easy to construct that means it’s cheap and easy to mock things out.

Next time
Running tests with Jasmine and your web browser

This is part of a series I'm writing for Agile+ on Google+ so you can follow as I post them there

Monday, November 26, 2012

Test Driven Development with Javascript

I've had a few opportunities to put my TDD skills to use in this brave new frontier. So I wanted to share what I've learned, what pitfalls to avoid and where the low hanging fruit is.

Why use TDD in Javascript?
We've reached a point where few people can develop an application without there being a web interface included. And now it's not enough to build a website with a few forms and buttons it has to be a "web application" with the detailed user interface and responsiveness we would expect of a desktop application. Javascript is serious business. Clients are not going to compliment you on your stable and fully tested backend if the UI is painful.

When should you not use TDD in Javascript?
Javascript has a strange distinction. It is the most widely used programming language in the world. It is also the most disliked programming language in the world. It can be hard to get programmers to learn Javascript before they dive into writing Javascript. So you cannot expect unit tests in that environment. It may be that your programmers are just copying the latest jquery plugins and performing wonders while writing very little Javascript  If there’s very little Javascript being written then there’s no point trying to automate the testing.

Okay. So do I test in Selenium or Watir?
Wait just a minute now! Those are not unit testing frameworks. They can be great for integration testing to see the behaviour with the DOM and a backend. But TDD requires unit tests that are fast to write and fast to run. I’m going to use Jasmine but there’s lots of other unit test frameworks to pick from and I’ll walk through the ways to run it on a console, through a browser and distributed through many browsers.

Next time
Writing testable javascript and what not to test

This is part of a series I'm writing for Agile+ on Google+ so you can follow as I post them there

Wednesday, September 12, 2012

Place to discuss Agile topics

I haven't been completely idle. Appart from my rambings on twitter and google+ I've also been making regular posts to a new facebook group on Agile Software Development.
I will return to talk about RestScriptFixture, fun with NodeJS and are games (like movies) out of new ideas?

Friday, May 25, 2012


So most of the tests I'm writing now in Fitnesse are using RestFixture. Being able to do all this black box style testing has helped me get a lot of tests up and running without having to change the existing code base. Now I've taken a step future with my own little fork so I can use scenarios and build nice BDD style scripts. But first I want to give me own quick guide to using RestFixture

Step 1: Installing
You can dive straight in by grabbing the latest jar files for RestFixture here
If you know what you're doing can get the nodep version to work nicely along side other libraries you may be including in Fitnesse. But I grabbed the 'full' version and unzipped it into a RestFixture folder alongside my FitNesseRoot folder.
Step 2: Write your first test
I took advantage of the built in Fitnesse api as a basic test and wrote a page called RestFixture with the following contents
!define TEST_SYSTEM {slim}

!path RestFixture/lib/*.jar
!path RestFixture/RestFixture.jar

!define expectedReturnHeaders {Content-Length : [\d]+
Content-Type : text/xml }

|!! | http://localhost:8080|
|GET|/RestFixture?rss|200| ${expectedReturnHeaders} |//title[text()='RestFixture']|
If that runs and passes then congratulations! You've got a working RestFixture install! Step 3: Profit!

Pitfall 1: &
Stuff like /loadpreferences?session=9999&user=admin&module=config would fail and I was scratching my head. Now any good RESTful api designer will tell you that's a stupid url to be using and they'd be write. But it's someone's old mistake and I can either tell them how stupid they are or appreciate the fact it works and move on trying to get everything that's broken working too.
Why did this fail? Well because Fitnesse was turning my & into & and messing up the url with nothing to parse it back. Solution: Surround the url with !- -! to stop Fitnesse doing any parsing on it and it'll work just fine.
Pitfall 2: json + xpath
I was trying to figure out how to handle json that didn't confirm nicely to use xpath commands. Unless your json is wrapped in something like {node: stuff} then the xpath stuff just doesn't work. Something I will try and fix at some point. So i used a lot of the javascript parsing in there to access the jsonbody variable that would work just fine. So I got lots of things like this:
/* javascript */ jsonbody == false;
or this
/* javascript */ 
var match = null;
for (var i=0; i != jsonbody.length && match == null; ++i) {
  var node = jsonbody[i].map;
  if ( == 'Bob') {
    match = node;
match != null;
or use the let ... js statements like this
Pitfall 3: Posting form like data
It's documented in later versions but not in 2.0 beta that you can use set body with url encoded data to behave like posted form data. Very handy. Although I'm still trying to figure out how this can work if combined with a file upload.

Wednesday, May 16, 2012

JDBC Fixture

I created this class so I can run basic database commands from within Fitnesse. Dropping SQL into a Fitnesse page not recommended but it can still be a useful tool for a few reasons:
  1. It is reusable so you can drop it into lots of tests without making new classes all the time
  2. It's a good bridging tool in trying to get developers using fitnesse who are not used to creating an abstraction layer for their tests
  3. It's handy to combine with RestFixture when you need to adjust or validate the data but cannot do it through a rest command
But again it's not recommended. Better to create a new fixture with a function like
public void createNewUser(String name, String password)

Here is what running the JDBCFixture looks like in the fitnesse
!path lib/*.jar

!|import                  |

!|script|DBQueryFixture|mydatabase                                           |user      |pass         |
|check  |statement     |UPDATE users SET city='New York' WHERE name='Big Joe'|1                       |

!|script|DBQueryFixture|mydatabase                                           |user      |pass         |
|query  |SELECT * FROM users WHERE name='Big Joe'                                                     |
|show   |get row       |0                                                    |and column|name         |
|check  |get row       |0                                                    |and column|city|New York|
So it's pretty straight forward. If you're already familiar with script tables you'll know how you can drop switch between check and show for when you just want to see the result vs when you want to validate it. There are probably lots of statements you'll want to run without looking at the number returned (number of rows modified or -1 if there is an error). My fixuture is hard coded to use a local postgres database (and the postgres jdbc jar is in my lib folder). But it's easy enough to change that for your own version
package com.warmage.util.fixtures;

import java.util.ArrayList;
import java.util.List;
import java.sql.*;

import static util.ListUtility.list;

public class DBQueryFixture {

    // constants
    final private String baseUrl = "jdbc:postgresql://localhost:5432/";

     * Constructor for script table
     * @param database - Database name
     * @param dbUser   - User to use
     * @param dbPassword - Password to use
    public DBQueryFixture(String database, String dbUser, String dbPassword) {
        try {
            jdbcConnection = DriverManager.getConnection(baseUrl + database, dbUser, dbPassword);
        } catch (SQLException e) {
            System.out.println("Error connecting to the database");

    public int statement(String query) {
            try {
                Statement st = jdbcConnection.createStatement();
                return st.executeUpdate(query);
            catch (SQLException e){
                System.out.println("Error executing statement");
            return -1;

    public void query(String query) {
        try {
            Statement st = jdbcConnection.createStatement(
                  ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
            results = st.executeQuery(query);
        catch (SQLException e){
            System.out.println("Error executing query");

    public String getCell(String column) {
        if (results != null) {
            try {
                return results.getString(column);
            catch (SQLException e){
               System.out.println("Error fetching cell for column " + column);
        return "null";

    public String getRowAndColumn(int row, String column) {
        if (results != null && row >= 0) {
            try {
                // Iterate to the right row
                boolean resultsValid = true;
                while(resultsValid && 0 <= row) {
                    resultsValid =;

                if (resultsValid) {
                    return getCell(column);
                } else {
                    System.out.println("Error row count too big");

            catch (SQLException e){
                System.out.println("Error moving through rows");
        return "null";

    private Connection jdbcConnection;
    private ResultSet results = null;