Agile testing strategies

21 Oct

One of the participants in a course asked me about test strategies for agile projects.

Drawing on my extensive testing background I responded that I had heard of them and that I had also heard they were probably a good idea.

But I think he may have wanted a more extensive answer.  He is doing test strategies on both waterfall and agile projects and his team are still using a long-winded document with a lot of cutting and pasting from previous projects.

He understands that strategy is more about the creation of the strategy than the publishing of a document and is comfortable with the idea that he collaborates and thinks before documenting the outcomes.  But he is looking for a more effective way to document the test strategy and maybe have a template or approach that works for his agile projects.

Can anyone give him better advise than my sage comment “I have heard it is a good idea to have one.”?

Posted by James King


Posted by on October 21, 2009 in Agile, Test Strategy, Testing


4 responses to “Agile testing strategies

  1. Sharon Robson

    October 22, 2009 at 10:06 am

    Hi James, I see Agile testing is just like “traditional testing” but done in different order.

    There are two key focuses – 1 from within the iteration and 1 from the overall testing of the piece of work.

    Outside of the iteration but in the overall approach to testing the work, to assist in the benefits of early testing, we often find the testers working an iteration in advance. They are working with the BAs and the Business to clearly define “testable” acceptance criteria. This is where the test plans can also be written and updated to include system testing – using stories and “use cases” that reflect real world usage scenarios. Planning must also be done for the show cases, and the deployment testing.

    The key things are to clearly highlight and understand the importance of the different levels of testing (unit, integration, system and acceptance) and to ensure that they are included, where relevant, in an iteration somewhere. This is usually spread throughout the overall piece of work.

    Regression is the biggest risk, so the team needs to be made hyperaware of it, and how to combat it – TDD and Automated Testing is the key here.
    There may be two types of automation though – one at the developer/unit test level that is run as part of the Continuous Integration activities, and another suite that is run by the testers to automate system test scenarios and to allow integration testing as well as regression testing from the system/user perspective.

    If performance testing (stress, load, capability etc) is a factor then provision must be made for when and how to incorporate these into an iteration, maybe a few specific performance iterations need to be run. These types of test usually require specific tools and approaches – they are however, able to be done early in the development of the product – they don’t need to wait until the end.

    Test data has to be defined and collected. One of the risks with an Agile approach is the lack of clear test data and a test environment beyond the dev team environment. There must be provision made for this from the overall project perspective.

    For within the iteration the test strategy must factor in:
    1. Automated unit testing
    2. Test driven development
    3. Smoke testing of the CI – no smoke testing passes, no testing
    4. Don’t change the automated Unit Testing – even if there is refactoring….the tests are the knowledge base of the system….
    5. Aim to be extremely regression averse!
    6. Be prepared to spend an iteration doing system testing as there may not be time within an iteration to do new feature testing, regression testing AND system testing

    This is just the start, but there are no fundamental differences in the testing strategy – it all still needs to be done! The difference is in the order, granularity and implementation. The techniques used will depend on the project under development.

    The big thing to remember is that this strategy can be a document or diagram. It doesn’t have to be long, nor detailed. The “strategy” is a high-level outline of “how” testing will be done within the piece of work…not the nitty-gritty details – that is done in a test plan (if required). I like to incorporate it as a row of notes/tasks below the Release/Iteration Plan on the wall. The tasks need to be visible so that they can be planned for and estimated.

    Hope this helps.

  2. bjosman

    October 22, 2009 at 12:36 pm

    Great question actually…..without understanding the *full* context of the course participants dilemma and that I think what he wants is a *different* approach to writing a test strategy/plan (as opposed to a more document/process focussed *traditional* plan, what i think he/she is after may be the following.

    I THINK what he/she is saying is what other ways can I devise a light weight, more efficient for my context test plan and can it be a different way of thinking about and presenting this plan?

    I went through a very similar exercise whereby I used James Bach’s Heuristic Test Strategy model ( formulate my test plan on an agile project. As a result, the plan lacked most of the preliminary *stuff* because we were collaborating a lot so the degree for miscommunication was somewhat negated.

    I thought the plan really cut to the chase and stated what need to be achieved (and on a philosophical note, I kept General Eisenhower’s quote on planning to the forefront of my mind “In preparing for battle I have found the plan to be useless but the planning was indispensible”.)

    I was very happy with my less is more approach. I think I had the plan down to less than seven odd pages (whereas the majority of my plans tended to be ALOT bigger and carried a lot more *waffle* – in my opinion). When I presented this to the PM, she asked to me to re-write it and copy and pasted from old *traditional* test plan which then blew the test plan out to something like 30 pages! I was not impressed until I realised an important lesson – understand the audience! I didn’t consider that – they were lawyers and as such, MORE is definitely MORE for them! A good lesson learned because in that context, *heavyweight* documentation was good 

    (But I still don’t see how it added any additional testing value over the original plan.) 🙂

    I found James Bach’s model to work well for me and again it depends on what your after, the context of your project and how comfortable you are in trying (and defending) something different!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: