Tuesday, June 6, 2017

What role are your tests serving.

We had a discussion at work after watching the excellent Kelvin Henney talk about good unit tests. And it helped me clarify the many roles tests play and why people get a habit of complaining about their tests getting in the way. So here is the four categories a test is trying to balance:


  • Requirements
  • Regression
  • Documentation
  • Refactoring guide


Most test today are written as requirements. And this is a huge improvement over the past when we'd write tests later and try and get code coverage. Those 'later' tests are useless. The requirement tests happen for high level acceptance tests and in low level TDD too. Your test defines a behaviour you want. Once your test passes you don't go back to change it at all beyond some obvious cleanups like code duplication. These tests allow rapid development and reliable progress. But they can hinder refactoring, provide poor feedback on failures for regression and may be too implementation focused to be readable.

And easy correction is to make tests act as good documentation. Unit tests should explain component behaviour to other programmers. Think of it as the examples section in your book that any programmer is going to flip to and read first. Higher level tests can use BDD, tables or other techniques to create highly readable descriptions that can be shared with your product owner or customer. One thing you'll notice if you try and make unit tests act as good documentation is you'll want to avoid the london-style or mockist style test where every secondary object is mocked out. All those mocks heavily hinder the tests being a document as it shifts away from being an example of how the code runs on the product.

You may think all those requirements focused tests are all you need to cover the regression case. But for good regression tests you want highly informative error messages. I find those get introduced later when you start seeing such failures. That might be when a bug is introduced or a new feature makes old tests inaccurate. That's a good time to go back through those assert statements and make sure they give enough information to know how to correct it. As once corrected those asserts will benefit you ever after. There are also some frameworks that can help like the highly informative asserts from py.test.

Unit tests are your best guide for refactoring. So you want a complete suite of tests that run very fast. That way you can change a line of code and know within a few seconds if that change is okay. Higher level tests are helpful too in that you could replace an entire component (perhaps something custom with something off-the-shelf) and see everything still works. Can you replace your database with confidence? That's the kind of feedback high level tests can give to help refactoring.

So a single test can satisfy all four of these needs. But it doesn't need to so it's better to figure out where the focus is for the test you are writing at the time. Some tests might only be there for documentation. It's okay for tests to evolve to help refactoring only when you need that refactoring. But don't assume that once a test is passing that it automatically ticks all these boxes. It's just a check at that point.

Thursday, May 25, 2017

When Agile is not common sense

It's been about 8 years since I got my Agile training from James Grenning and started an Agile Pilot with my team. Right from the start I was super keen and got people practising TDD and Pair Programming even though I didn't really know if that effort would pay off. I just knew we had lots of problems with the old way so I wanted to embrace the new.

So I'd run my own little sessions with the broader team in Sydney sharing Agile concepts and ideas. Giving us a chance to discuss things and keep learning long after James had left. But often I would present new ideas and a few people in the group would respond "oh of course, that's just common sense". Initially I thought it was great as they were embracing these ideas as valuable even though it was not what we were doing before. But after a while I started to get frustrated because they we're not really listening. If some small part of a concept fit nicely with their "common sense" then they took that on and ignored the rest.

For example I'd point out that Agile asks us to plan regularly but to never put too much value into any plan. You need to be prepared to throw out your plan from yesterday because you learned something new today. "Oh of course, that's just common sense." But what I'd observe is they didn't re-plan regularly, they would just stick to the old plan unless something was clearly going wrong, and only then would re-plan in an attempt to react. They didn't adopt the new idea as they didn't understand that it's about being proactive. They didn't understand the value in making continuous minor adjustments or even just discussing how their understanding has shifted over time.



So I did a presentation to them about when Agile is not common sense. Put out the ideas that clearly challenged their old thinking and through that made my case that they allow other ideas to challenge them too. My list has grown over the years and it's still something I refer to when I feel people are being to complacent.

TDD
We all understand the value in having tests. But writing a test before there is any code? Writing a test before we have a clear picture of what we're building? TDD should challenge the way you learned to program.

Pair Programming
Two heads are smarter than one. But we all assume it'll be the same speed as just 1 person so under value the benefits. But in studies we find that, with practice, the speed increases by quite a lot. Most of programming is not typing it's thinking about the problem. The same is true for creating tests in that typing is a very small part of the time spent. With practice we see pairs operate at about 1.8x the speed of an individual while also having 50% fewer bugs.

Mob Programming
See pair programming

No Estimates
Estimates can be a dysfunction. Is someone asking for a completion date or trying to figure out a cost, plan a release or looking for a deadline to enforce. Move the conversation away from estimates as much as possible.

If something is difficult or error prone then do it more often
Teams will often avoid doing a minor release due to headaches in the release process. A good coach will then insist that releases be daily until release issues are resolved. It applies to many areas where we avoid facing issues rather than fixing them.

Maximise the work not done
We want to work on a prioritised list so that we can avoid doing things that have little value. Sure the customer says they want A, B & C but if C will help them immediately then do that and give them the software before working on A & B. You might find out that with C fixed that B drops from being the second most important feature to no longer being required.

Safe to fail
Encourage teams to challenge themselves and occasionally fail. If the team never fails then how do you know they are actually trying to improve? If treat everything as so important you cannot risk failure then you do not allow improvement. So you are not allowing teams to operate at their best when you demand only the best.

Having a detailed and complete test suite allows quicker changes to your code
Many with only a little experience at writing tests would disagree. The tests break in confusing ways. Changes to the software trigger many required changes in the tests. But really that's just changing code with poor tests that are too tightly coupled and fail in bad ways.

That's my list for now. As an aside it's worth remembering that common sense is cultural and really just wraps up a lot of assumptions we share. I do this contrast more to challenge people's thinking than really seeing a lot of value in something being common sense or not. When we start thinking of something as common sense we assume others know it and that's an unhelpful assumption.

Thursday, May 11, 2017

Applying the Toyota Kata

Last year I got to attend a workshop on the Toyota Kata by Hakan Forss and I could immediately see how it could be useful but I didn't have an obvious place to apply it until recently.

So what is the Toyota Kata? I see it as an alternative to retrospectives. It lets you define a collection of goals and run experiments to see if you can get progress towards that goal. Unlike retrospectives you can define goals that may seem distant (release to customers on every passing commit), may be simply something you want to maintain (have zero bugs in production) or be impossible (have zero wait time in moving a story from backlog to done). Once you've built this collection of measurable goals you decide what you're going to try and your expected outcome (we will to TDD and have 50% less bugs in production). You can define what the period is before you check in again and decide on the next step. But as with anything Agile; shorter is better. You can work it into a normal two week scrum cycle or if improvements are important enough then spend a small time on it every day. Next time you get together you can assess how the experiment(s) did and what you want to try and do next.

I really like how it's focused on doing small experiments rather than committing yourself to something to fix the problem immediately. Especially for those big issues it frees you up to try anything because you'll know soon if it helped and can try something else. With normal retrospectives it can be hard to get people to conduct experiments.

So my team recently went from doing 2 week sprints to a Kan-ban style flow of work. I had some concerns that it could lead to a range of problems and wanted it to be a positive change, So we figured out some measurable goals where we wanted improvement and things we to maintain. The improvement ended up being focused around cycle time as I felt sprints encouraged us to get things done because a sprint was ending even though it's an artificial deadline. If a task lasted more than a sprint we were able to quickly see it as an issue. Other measures were around velocity (measured in weeks instead of sprints) and team comfort (do they like the change or would they rather go back).

So far things have been great. Our cycle time is getting better and everyone is comfortable with the process. It's not a clear winner where people love the new process over the old. And our goals for cycle time seem lofty compared to what we ended up measuring. But it's given us a structure to keep experimenting and keep chipping away at these things. I absolutely would recommend it to anyone facing issues they feel cannot be resolved in a normal retrospective cycle. It can compliment or replace retrospectives as you see fit.

Wednesday, May 10, 2017

So I've been advised to create a Coaching Manual

I'm trying to build up my skills as an Agile Coach. In a way I've been doing this already now for about 8 years. But it's getting a bit more structure and organisation.

The way I see it learning is part of software development. It always has been. Some people get comfortable with their knowledge and workload and stop learning but that has never been me. When I get too comfortable I start looking for something else to do.

But my learnings are just a jumble in my brain. Sometimes it impresses me as I can pull out a name, a quote or apply a process to something specific. And sometimes it lets me down so I have to revisit and revise like I'm doing right now. I'm listening to Esther Derby talk about her Six Rules of Change and writing this at the same time. Not something I recommend but I just want a refresher as I've studied it before. I'll probably grab the slides later and use it as part of a mini-presentation. Sorry you just ran into my jumble.

What I'm trying to say is it's time to organise that jumble into something written. And the best place I can think of to write it is here. So I'll try and every week regurgitate something out of my head and into words. Usually it'll be referencing someone else smarter than me rather than anything terribly unique. But that's okay as it'll form a nice reference I can go back to for myself.

Thursday, January 5, 2017

Things I’ve learned from running lean coffee and why you should organise one too

I’ve been running a 30 minute lean coffee session in my workplace every two weeks for 18 months now. Our first meeting for this year is tomorrow. I started the sessions when a bunch of people from work went to the Agile Australia conference and we wanted people to keep sharing ideas and other learnings. Now I invite any software developer in the building so my invite list has grown to 99 people but I typically only get about 10.

How lean coffee works: A few people from work got to take part in one at Agile Australia but I missed it myself. So I just got online and read everything at http://leancoffee.org/ I grabbed a bunch of index cards and texters and was ready to go.

I feel it’s super important to my company as it’s the only time people from all the software development teams can meet each other and discuss things. It’s helping break down walls between teams and helping ensure learnings can spread to other teams quicker.

  • Location matters. I started off in the building’s cafe but it would get too noisy when the coffee grinder is going. So we moved to another table in the entryway. I like that we’re highly visible but also in a casual environment. I find meeting rooms too formal. I also like that there are lots of tables so I can split the group up on demand.
  • I setup up one table for the discussion and if that table gets full I ask someone with experience to help run that table and move to a new one. I try and split like this once we hit 8 people. Timing of topics is done on a phone so anyone can do it and once someone has come a couple of times they know the routine and can run a table that has already been setup.
  • I am always prepared to throw in a topic to get things started. I especially like light hearted subjects. Last time my card was “How would Santa write software?”. This is extra useful when I’m going to leave a table to setup the next one.
  • Be a good host and facilitate the discussion. If you see someone get cut off ask what they were going to say at a later moment.
  • Take notes. You never know when you’re going to learn something. But also you might get useful feedback for the lean coffee at unexpected moments. Who else should I invite? What’s a good topic for next time? Do we need to change the location? The time?
  • I run mine for 30 minutes as it’s pretty hard to get people to leave their desk when I’m not supplying lunch. But at 30 minutes we are lucky if we can get through 4 topics. You could do it an hour but I think the energy and attention would quickly drop after that. Also I host mine in the morning as I find it easier to get people to make time just before their standup meetings.
  • I only get a few people attending. And that’s okay but I try and push to get people from different teams coming. It’s all about sharing information and if it’s only a couple of teams then there is a huge part of the company we’re not hearing from.
  • Encourage people to bring coffee and tea. I like the casual feel it helps add.
  • I use index cards instead of post its. I find it works better on a flat table and means I’m less likely to have a texter bleed through and mark the table.

Friday, August 14, 2015

Can Programmers Test?

I spend a lot of time working closely with programmers on the software testing process. Much of my career mission is to get more people embracing techniques like Test Driven Development. As I push more and more of the testing demands onto the developers I sometimes run into people who say they need a tester for "independent testing".



It's a concept I'm familiar with from early days in my career. A tester can look at the system and requirements with fresh eyes and find all those cases the programmer didn't consider. If the programmer does all the testing they'll be too tempted to just test the cases they know the code is handling and not discover any new areas where bugs may be hiding. The problem I have with this idea is I never was that type of programmer.

As a programmer if I didn't test something it was purely out of laziness and arrogance. I'm a big supporter of Larry Wall's three great virtues of a programmer. A lazy programmer will do their best to eliminate bugs as early as possible to save time. But if the programmer knows someone else will spend a lot of time looking for bugs then why should he bother? Just chuck it to the test team as soon as possible for all that information. It takes either removing that test team or a higher level of discipline to break that pattern. The independent tester is not catching bugs the programmer would not have found himself. They're spending a lot of time documenting bugs the programmer couldn't be bothered finding himself.

The comparison I like to make is a writer and an editor. As a writer you need to stay focused and complete your work before you get to thinking about editing. Then you edit it. You edit it multiple times constantly revising your work until you think it's fantastic. Only at that point do you hand it over to a professional editor. I'd like to see programmers all do the same thing. You can do testing. You should do testing. You should test until you feel there are no bugs left to find. Then hand it to a tester and maybe they'll find one or two things.

Testers are not all that independent anyway. They're reading the same requirements as the programmer. They're under similar deadlines and goals to get the product out the door. Any argument that says a programmer would avoid testing something could just as equally be applied to a tester. The reality is it doesn't happen because we're professionals and understand the need for a quality product. There's no reason the demand for quality should be solely the responsibility of the tester.

What is required to get programmers more testing is education. They're not often taught about testing methods or the kind of data that makes for good tests. That alone would add great value to the tests they can quickly perform before handing over a product. But there also needs to be a shift away from the thinking that a programmers time is too valuable to perform testing. If the programmer finds the errors they can quickly fix them before moving onto their next task. The longer it takes before the bug is found the longer it takes for a fix to apply as the programmer has to open up old code, go through any required reviews again, and finally get the fix out for re-testing.

So I strongly believe programmers can and should test. But how do I get that message across to everyone?

Monday, August 3, 2015

How to Write Code Fast

Start with a clean slate with simple goals

Starting from scratch is such a rarely privilege in professional programming. We get to do it all the time when we’re learning and trying things out. In our early years getting stuck and stumbled on the most trivial challenges. And later being proud of the frameworks and architectures we can build in days (sometimes hours).
But there is rarely any money to be made in building something from scratch. There is no value in doing something trivial because it likely has already been done a thousand times before. So we need to add something to existing code. Maybe it’s something we already developed ourselves or something we bought. If we’re lucky the existing code is modular enough that the new feature can feel like starting with a clean slate and give us that burst of speed.
So you will often hear the cries from programmers that are working with that monolithic and aging code; “please, please let us rewrite this code from scratch”. I’ve made this cry a few times in my career. Sworn to make it fast enough to reduce the cost. Promising to fix all those bugs that we now endure. Brazen enough to believe there will be no new bugs introduced this time that cannot be plucked out. We lie so much. We lie to ourselves and to the company. We lie because the pain of that old code is so real and so present.
The reality is that recreating something from scratch often takes just as long as the original did. Which may be years. Is in a race to catch the original is still being patched and enhanced in use. And the switch is still horribly painful as you have spent so long writing code before it’s put into real use and only then you find a bunch more issues. So you blow the budget, often reintroduce many of the same bugs as before, sometimes a few new ones and now hate the new code so much you cry “please, please let us rewrite this code from scratch!”
So if we’re lucky we get to work on something simple from scratch. No wonder people work for startups for stock options as payment. Or if there is an existing modular system it can feel like working from scratch. So we see such frameworks are massively popular in your language of choice. But that speed may not last.

Start with high quality code with simple goals

A system that is highly modular can feel like working from scratch. It can do this by actually splitting any operation into a collection of lots of smaller applications. The unix operating system is built on this principle. No single program does much at all. But they all implement the same basic method of handling input and output. That allows us to pipe data into and out of any combination of these programs to perform increasingly more complex tasks. There is no motivation to write a new text sort program because one already exists. There is no motivation to write a new text sort function because it is so easy to call the existing program.
But additionally the code can be very well understood and structured in a way that it is easy to change or extend. When an existing code base creates a high confidence on the outcome of any change the programmers are free to change it in any way they wish. So that existing code becomes a nice launch pad to very quickly create a new variation or addition. It may not feel like working from scratch because you may be changing code rather than writing new files. But it allows small changes to be quickly released, tested and reviewed. So progress kicks off at a blistering speed.
But each change may bring with it some technical debt. Create a new link between two modules that were previously unaware of each other and you have quickly created a nice new feature. But you’ve also created a dependency that is not obvious. Now all the tests need to be aware that one affects the other. Any new changes to either module needs to be intimately aware of this relationship that got hacked in. Further features and changes create a complex web of relationships. It starts to lose that feeling of simplicity and confidence. Then the bugs creep in. The bugs you’re afraid to fix because it might break something else way over there.

Pile up technical debt (aka make a mess)

But the customer hasn’t noticed that bug yet. They just want features. It’s the new features that make money. It’s the new functionality that will fight off the competition. As long as the system holds up and functions right now there’s no time free to fight the bugs. Nevermind clean up the code and make it more modular. We want to go fast right?
I like the analogy of the application being like a restaurant. You can present a very nice front area with great service and a wonderful menu to the customers and make a lot of money. Little do they know the mess being made in the kitchen as the chefs franticly create meals. If the customers saw the mess, the pests and the rotting smell back in the kitchen they would never eat here. None them have gotten sick… yet.
You quickly get lots of features out and the customers love the product. Sure there’s a few bugs but they’ll endure it for all the value the application is giving them. But the demand for features doesn’t slow down. Each new feature is harder to do and risks introducing bugs as fragile code is pushed beyond its original function. Outside of the programming team there are murmurings. What happened to this fast pace the programmers used to manage? How did a good team become so bad?

Keep it clean and lean

So what’s left? It’s not financially beneficial to start from scratch. Even if we start from a base of quality it’s easy to lose our way and lose that base. What allows us to go fast and stay fast? Is technical debt as inevitable as entropy. Would the second law of programming read: The sum of technical debt in software increases as the code is changed and added to?
Robert C. Martin says, “The only way to go fast is to go well”. While there are plenty of other opportunities to write code fast many of them do not last. The lasting methods embrace the key practices of eXtreme Programming (XP). Pair programming, Test-Driven Development, Design Improvement, Continuous Integration and Collective Code Ownership. These are technical practices and may not mean much to customers or project managers. There are many people in a successful business that won’t be looking at the code quality. So the responsibility lies with the programmers as only they are reading and writing code each day.
Pair programming is easy to roll out but often resisted. Many programmers learned the complex world of computing as the complex world of social interaction seemed too daunting. But code reviews afterwards are a weak substitute. By then it’s far too easy to say the design is locked in now and too hard to fix. Code reviewers understanding of the solution in place is limited while pairs discover the solutions together and weigh up the options each can bring before progressing. There are many studies with good methodologies that show pair programming can dramatically reduce the number of bugs in the final product. If you care about quality the question should be; why are you not doing pair programming already? And now many are taking it even future with mob programming getting a whole team mind share.
Test-Driven Development (TDD) drives you towards a testable solution. Highly tested code is code that brings high confidence in making changes. High confidence in making changes means even late changes are easy and welcome. High confidence in making changes gives you that launch pad to make changes quickly and get results quickly. But you only get that launch pad if you were doing TDD all along. An ideal time to have started writing these tests was ten years ago. But you can settle with starting today. In the long run TDD will make you faster for a very small cost in time (10-20% according to studies). I would argue the cost is even less for complex tasks as it gets you writing code while you’re still figuring out the full solution. And now you can add to it in so many other ways with acceptance testing, behaviour driven development and more.
Design Improvement is a recognition that the system you’re building now many not be the same system you’re building in three months. The requirements will change and evolve and so too should the design. With the high knowledge of code we have thanks to pair programming and the high confidence in making changes thanks to TDD we are free to make changes that improve the overall design after the implementation. You can change the entire architecture of your code base piece by piece to eventually end up somewhere closer to the product you have now and not the one you intended six months ago.
Continuous Integration is to help recognise problems early and fix them early. If someone introduces a bug there is a big red flashing marker to stop work until it’s fixed. Much like you would expect on a factory production line when an issue introduced is noticed. The line is stopped and the problem is resolved immediately. And why stop there? You can get continuous releases going for rapid feedback from all your customers.
And finally Collective Code Ownership is critical to keeping any team of developers working fast. If your database expert is sick for a month you cannot afford to have all production slow down. If a reports system was developed by someone who since left the company how do you explain to the customer why further additions are coming so slowly? Pair programming goes a long way to resolving this but so does the TDD and design improvements. Keeping the system simple and one that the whole team intimately understands. If a developer sees a line of code they want to fix they should feel confident in fixing it, on the spot, without consulting anyone beyond the pair they are working with.
Too often Agile is rolled out without these core XP practices and while teams are releasing in regular cycles release speed is slow. Many just accept the slow buggy releases as inevitable in software development much like they did with the waterfall code of the 90s. So great is the possibilities of going fast and there is so little to lose. Asking people to put higher quality into the code and their development team should not be considered a cost. And thanks to that quality you can write code faster.

References:

The Costs and Benefits of Pair Programming. A Cockburn & L Williams.
On the Effectiveness of Unit Test Automation at Microsoft. L Williams , G Kudrjavets & N Nagappan.
Implications of Test-Driven Development A Pilot Study. R Kaufmann & D Janzen.
http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1038&context=csse_fac