Author Archives: bjosman

About bjosman

Principal Consultant at OsmanIT

Lessons learned in software testing – an experience report

In the google group Software Testers New Zealand, a discussion thread was posted about testers favourite testing books. One of the main books mentioned is Lessons learned in software testing.

Cynthia, a tester based in Auckland New Zealand wrote this great experience report using the book Lessons learned in software testing (quoted verbatim with permission).

“On 9 April 2010 08:33, Cynthia wrote:
It seems like everyone is recommending Lessons Learned in Software
Testing! I do too.
I have it on my desk and dip into it quite often.

I thought I would give an example of it being useful to me in a practical way.
Yesterday I found a bug, quite late in the development. It was quite an obvious one too, once you’d seen it. But it took me ages to “see” it.
I was kicking myself for not seeing it earlier, and thought I’d review how I missed it for so long, so to refresh my mind and give a different focus I glanced through the chapter “Thinking Like a Tester”.

Wow, there it was, Lesson 41 – “When you miss a bug, check whether the miss is surprising or just the natural outcome of your strategy”.

I realised that my strategy had been a bit “off”. The function I was testing was to provide some mathematical and date information to our client’s customers. I’d spent ages making sure the dates and calculations were correct.
But I hadn’t asked the right question – what was the reason our client wanted to give their customers this information? What was the problem they wanted us to solve for them by providing this function?

The bug was that calculation results were being presented in different numerical formats in different places – as 3-decimal places, as 2- decimal places and as integers. They were all correct. But they looked confusing, especially when rounding came into play.

And one of the problems our client wanted us to solve was to reduce the number of phone calls at their Call Centres from confused customers.(This was not documented in their Requirements, but had been discussed informally).

So I finally “saw” that this correct data didn’t solve their problem, when it was presented that way.

Now I know that I need to add into my strategy for each new bit of  functionality that I work on – always ask the question “What is the real problem we’re trying to solve here?”

Does anyone else have stories about how their favourite testing books help in real life?


What a fantastic experience report – thank you for sharing Cynthia and thank you to the authors of Lessons learned in software testing – Cem Kaner, James Bach and Bret Pettichord.

Leave a comment

Posted by on April 12, 2010 in Test Planning, Testing


Tags: ,

Pair Testing

I was reading Sharon’s post on Test Pairing (run by Karen N Johnson at STANZ 2009) and I automatically made the connection with Pair Testing. They aren’t the same and in talking with Karen at STANZ I mentioned my assumption to Karen which in turn reminded me of a time in which I was involved in pair testing a public Government website.

The great thing was that we had management support. Without it, It would’ve been easy to suggest that its a waste of time having two testers assigned to the same piece of work.

Huh? Two testers, same work? What’s that all about?

Pair testing isn’t new and in fact it has more well known cousin in Pair programming. What it does well is that it allows for two different sets of eyes testing the product. Essentially one is the *driver* and drives the keyboard whilst the other is the *observer* or *navigator* (I prefer navigator – its name suggests a more hands on role.) The navigator is there to suggest ideas, learn, be mentored, mentor, record, find information etc.  These roles are of course, interchangable, and allows for a very shared, creative testing activity (of course there are challenges as there are in any approach but we won’t discuss them here.)

As we were testing, we interchanged a number of times and discussed and discovered a number of bugs. This is one of the benefits of pair testing in that brain can be engaged on task longer and is a lot fresher because of the role swapping, different perspectives and the exchange of ideas they will come flowing.

I’ve been involed with pair testing and it works. I have also seen it in action and its an approach that we should consider more often!

Posted by Brian Osman

Leave a comment

Posted by on November 27, 2009 in Testing


Tags: ,


Following on a from two very successful conferences SDC and STANZ 2009 (in both Wellington and Sydney), SDC 2010 has been annouced with the theme – Business Analysis gets agile. This will no doubt be a fantastic conference! Start planning to attend now!

See SDC 2010 for more details!

Leave a comment

Posted by on August 28, 2009 in Uncategorized


With Freedom comes Great Responsibilty

Whatever life holds in store for me, I will never forget these words: “With great power comes great responsibility.” This is my gift, my curse. Who am I?

I’m Spiderman.

As with Peter Parker, I see the same holding true with agile testing. I was recently delivered a course on agile testing and I found myself repeating the above phrase but substituted the word “power” with the word “freedom” – “With great freedom comes great responsibility.”

I see agile as an adaptable framework or more accurately, a set of principles, that allow testers (amongst the entire project team of course) to look outside the “box” (testing processes) and yet at the same time let the framework help guide the testing activities.

 There are of course, certain constraints within an agile project (particularly the duration of an iteration) which means that as a tester, you need not only have to be flexible but also efficient.

I have been on far too many projects that have followed a traditional pattern (analysis, design, code, test – wait that sounds almost like a waterfall) that, in hindsight, weren’t the most effective way of building software.

 Some succeeded, alot failed.

 One particular BUFD (Big Up Front Design) project had a physical separation of the core team (Analysts lived in one area, testers in another and development in another city). There was a distinct lack of collaboration let alone co-operation and a degree of animosity between testers and the analysts. There were numerous changes to the specification within a three month period which meant a re-write of a majority of test scripts (literally thousands of test scripts) – all this without even seeing a working prototype.

This project failed – it was a death march from the beginning which was obvious in hindsight.

 I wonder if this project would’ve fared better if it was developed iteratively? Who knows? Maybe we could’ve discovered the problems earlier (and abandoned the project earlier?) or maybe the physical separation was too big a huddle to overcome?

 Many questions with many hypothetical answers.

 This project suffered, from a testing perspective, from inflexibity. We all wrote test scripts (approximately 14 testers in the team) which kept changing. We adapted but we were no where near efficient. We clung to the “box” (the teams testing process) because that was all we knew. There was no great responsibility because the responsibility was mandated by the “box”.

 From an agile project perspective, the walls on the “box” can change quite quickly – one minute it’s a box, the other it’s a cube. This can be disconcerting for some for it appears that the fluidity of the “box” is chaos but what it really is, is freedom and “with freedom comes great responsibility!”

Good agile, agile done well appears to have a greater degree of freedom in the way testing is approached and performed but with this freedom comes disciple, structure, governance – this is the great responsibility. We value working software yet we don’t sacrifice goodness for sloppiness, constant velocity for speed or good practises for dogma.

 As testers, we understand that the project context helps determine our approach, the freedom to achieve this is based on the context which in turns helps us understand the responsibility we have in delivering our part of the bargain with the project team.

Posted by Brian Osman


Posted by on July 6, 2009 in Agile, Testing


Tags: , , , , , , , ,