Friday, May 25, 2012

RestFixture

So most of the tests I'm writing now in Fitnesse are using RestFixture. Being able to do all this black box style testing has helped me get a lot of tests up and running without having to change the existing code base. Now I've taken a step future with my own little fork so I can use scenarios and build nice BDD style scripts. But first I want to give me own quick guide to using RestFixture

Step 1: Installing
You can dive straight in by grabbing the latest jar files for RestFixture here https://github.com/smartrics/RestFixture/downloads
If you know what you're doing can get the nodep version to work nicely along side other libraries you may be including in Fitnesse. But I grabbed the 'full' version and unzipped it into a RestFixture folder alongside my FitNesseRoot folder.
Step 2: Write your first test
I took advantage of the built in Fitnesse api as a basic test and wrote a page called RestFixture with the following contents
!define TEST_SYSTEM {slim}

!path RestFixture/lib/*.jar
!path RestFixture/RestFixture.jar

!define expectedReturnHeaders {Content-Length : [\d]+
Content-Type : text/xml }

|!-Table:smartrics.rest.fitnesse.fixture.RestFixture-! | http://localhost:8080|
|GET|/RestFixture?rss|200| ${expectedReturnHeaders} |//title[text()='RestFixture']|
If that runs and passes then congratulations! You've got a working RestFixture install! Step 3: Profit!


Pitfall 1: &
Stuff like /loadpreferences?session=9999&user=admin&module=config would fail and I was scratching my head. Now any good RESTful api designer will tell you that's a stupid url to be using and they'd be write. But it's someone's old mistake and I can either tell them how stupid they are or appreciate the fact it works and move on trying to get everything that's broken working too.
Why did this fail? Well because Fitnesse was turning my & into & and messing up the url with nothing to parse it back. Solution: Surround the url with !- -! to stop Fitnesse doing any parsing on it and it'll work just fine.
Pitfall 2: json + xpath
I was trying to figure out how to handle json that didn't confirm nicely to use xpath commands. Unless your json is wrapped in something like {node: stuff} then the xpath stuff just doesn't work. Something I will try and fix at some point. So i used a lot of the javascript parsing in there to access the jsonbody variable that would work just fine. So I got lots of things like this:
/* javascript */ jsonbody == false;
or this
/* javascript */ 
var match = null;
for (var i=0; i != jsonbody.length && match == null; ++i) {
  var node = jsonbody[i].map;
  if (node.name == 'Bob') {
    match = node;
  }
}
match != null;
or use the let ... js statements like this
response.jsonbody[4].block.name
Pitfall 3: Posting form like data
It's documented in later versions but not in 2.0 beta that you can use set body with url encoded data to behave like posted form data. Very handy. Although I'm still trying to figure out how this can work if combined with a file upload.

Wednesday, May 16, 2012

JDBC Fixture

I created this class so I can run basic database commands from within Fitnesse. Dropping SQL into a Fitnesse page not recommended but it can still be a useful tool for a few reasons:
  1. It is reusable so you can drop it into lots of tests without making new classes all the time
  2. It's a good bridging tool in trying to get developers using fitnesse who are not used to creating an abstraction layer for their tests
  3. It's handy to combine with RestFixture when you need to adjust or validate the data but cannot do it through a rest command
But again it's not recommended. Better to create a new fixture with a function like
public void createNewUser(String name, String password)


Here is what running the JDBCFixture looks like in the fitnesse
!path lib/*.jar

!|import                  |
|com.warmage.util.fixtures|

!|script|DBQueryFixture|mydatabase                                           |user      |pass         |
|check  |statement     |UPDATE users SET city='New York' WHERE name='Big Joe'|1                       |

!|script|DBQueryFixture|mydatabase                                           |user      |pass         |
|query  |SELECT * FROM users WHERE name='Big Joe'                                                     |
|show   |get row       |0                                                    |and column|name         |
|check  |get row       |0                                                    |and column|city|New York|
So it's pretty straight forward. If you're already familiar with script tables you'll know how you can drop switch between check and show for when you just want to see the result vs when you want to validate it. There are probably lots of statements you'll want to run without looking at the number returned (number of rows modified or -1 if there is an error). My fixuture is hard coded to use a local postgres database (and the postgres jdbc jar is in my lib folder). But it's easy enough to change that for your own version
package com.warmage.util.fixtures;

import java.util.ArrayList;
import java.util.List;
import java.sql.*;

import static util.ListUtility.list;

public class DBQueryFixture {

    // constants
    final private String baseUrl = "jdbc:postgresql://localhost:5432/";

    /**
     * Constructor for script table
     *
     * @param database - Database name
     * @param dbUser   - User to use
     * @param dbPassword - Password to use
     */
    public DBQueryFixture(String database, String dbUser, String dbPassword) {
        try {
            jdbcConnection = DriverManager.getConnection(baseUrl + database, dbUser, dbPassword);
        } catch (SQLException e) {
            System.out.println("Error connecting to the database");
        }
    }

    public int statement(String query) {
            try {
                Statement st = jdbcConnection.createStatement();
                return st.executeUpdate(query);
            }
            catch (SQLException e){
                System.out.println("Error executing statement");
            }
            return -1;
    }

    public void query(String query) {
        try {
            Statement st = jdbcConnection.createStatement(
                  ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
            results = st.executeQuery(query);
        }
        catch (SQLException e){
            System.out.println("Error executing query");
        }
    }

    public String getCell(String column) {
        if (results != null) {
            try {
                return results.getString(column);
            }
            catch (SQLException e){
               System.out.println("Error fetching cell for column " + column);
            }
        }
        return "null";
    }

    public String getRowAndColumn(int row, String column) {
        if (results != null && row >= 0) {
            try {
                // Iterate to the right row
                results.beforeFirst();
                boolean resultsValid = true;
                while(resultsValid && 0 <= row) {
                    resultsValid = results.next();
                    row--;
                }

                if (resultsValid) {
                    return getCell(column);
                } else {
                    System.out.println("Error row count too big");
                }

            }
            catch (SQLException e){
                System.out.println("Error moving through rows");
            }
        }
        return "null";
    }

    private Connection jdbcConnection;
    private ResultSet results = null;
}

Sunday, May 13, 2012

Skipping over fitnesse tutorial

I was thinking of writing a short bit on writing tests in Fitnesse. But instead I'm going to recommend the tutorials by Brett L. Schuchert http://schuchert.wikispaces.com/FitNesse.Tutorials as they skip straight into using SLIM. The guides on fitnesse.org (and with the install) are good but explain everything with FIT first before racing though what is different with SLIM. That's just the nature of how Fitnesse was developed but it's worth skipping straight to SLIM as it has less requirements and can be more flexable.

Saturday, May 12, 2012

Setting up Fitnesse on Ubuntu in 7 steps

Some pretty basic steps but just to make sure it's here for everyone to see. Setting up fitnesse and running the jar is easy enough. Just go to http://fitnesse.org/ and get started and do it on your desktop just to see it in action. But for me that wasn't good enough I wanted it to run as service on ubuntu.

I stole a few tricks from how ubuntu runs jenkins and setup fitnesse a similar way.

1. Create a user and group for fitnesse (optional)
I didn't do this because I wanted tomcat, jenkins and fitnesse all running as the same user. Call it laziness to avoid any permissions classing but it doesn't change the process that you need to create or choose what user you're going to make it run as. Don't make it run as your user or root!

2. Download the jar file and place it in /usr/share/fitnesse
Make the folder too of course. It can belong to root as long as the fitnesse user has read access

3. Create the folder to run in at /var/lib/fitnesse
Fitnesse user needs write permissions here so may as well make it the owner. This is where FitNesseRoot is going to end up and also where you're going to want any libraries and classes it can include. I found fitnesse isn't very good at using absolute paths when doing includes and also doesn't handle spaces. Something to look out for!

4. Create the folder to store fitnesse logs /var/log/fitnesse
Make sure the fitnesse user can write
5. Create the below file as /etc/init/fitnesse.conf
description "fitnesse: Fitnesse acceptance testing framework"
author "geoff@warmage.com"

start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel [!2345]

env USER="fitnesse"
env GROUP="fitnesse"
env FITNESSE_LOG="/var/log/fitnesse"
env FITNESSE_ROOT="/var/lib/fitnesse"
env HTTP_PORT=8081
env JAVA_OPTS=""
env JAVA_HOME="/usr/lib/jvm/default-java"

#limit nofile 8192 8192

pre-start script
    test -f $FITNESSE_ROOT/fitnesse.jar || { stop ; exit 0; }
end script

script
    FITNESSE_ARGS="-p $HTTP_PORT -l $FITNESSE_LOG"
    exec daemon --name=fitnesse --inherit  --chdir=$FITNESSE_ROOT \
        --output=$FITNESSE_LOG/fitnesse-output.log --user=$USER \
        -- $JAVA_HOME/bin/java $JAVA_OPTS -jar fitnesse.jar $FITNESSE_ARGS
end script

6. Link from /etc/init.d/fitnesse to /lib/init/upstart-job
sudo ln -s /lib/init/upstart-job /etc/init.d/fitnesse
Make sure it's executable
sudo chmod +x /etc/init.d/fitnesse
nearly there now

7. Setup run levels for fitnesse to run with
I used sudo sysv-rc-conf -P to see the run levels and turned on 2, 3, 4 & 5. Could probably have dropped 2 but whatever, that's what most use. There seems to be lots of other ways to do this but this is what I did.

That's it! I'm probably forgetting something and I'm sure there's better ways to do some of the steps. There's lots of guides out there if you search 'ubuntu daemon' but knowing the quirks of fitnesse (spaces in folder names? where do you want files?) is still relevant.

Thursday, May 3, 2012

Fitnesse

Anyone who has worked with me in the last 3 years knows I have a programmer crush on Uncle Bob and I got most of my initial training on Agile development from James Grenning. So it was just a matter of time until I started using Fitnesse. Now fitnesse (and fit originally) is an interesting tool to try and get acceptance tests OUT of code. After all why should programmers (or programmer-ish testers) be the only ones to write, read and review these tests? Acceptance testing is about making sure you deliver what the customer wanted so shouldn't the customer and other management people be reading them too? So fitnesse meets them in a middle ground. A wiki.

A wiki is a good place to go because you can get a manager or customer to read a wiki. You can, with a bit of a push, even get them to start contributing to a wiki. Good luck trying to get them using jmeter or read what you created in cucumber! Those are great tools but they don't scratch the itch that fitnesse was written for.

So I've been writing fixtures and playing around with scenarios and scripts. But what has me really excited is combining all that with RestFixture. I'm currently dealing with a lot of java code with all the front end piped through a RESTful service. It's a pretty common way to develop any distributed service these days and works pretty well. So I've been able to build up detailed tests that push all kinds of scenarios through the rest api without touching any UI code. And I've been able to condense it down to user stories that read like English so I can get the business analysts types to read and review them. With a bit of training I might even get them writing the first drafts of new areas to test.

But it hasn't been without challenges. So I want to document exactly what I've done and the kind of tests I'm able to now write. More post will come later but first I need to clean up my fork of RestFixture so I can get my changes accepted back into the original!