For me one of the key questions about the Standish Group survey is their definition of “failure”.
I feel that cancelling a bad project early is actually a good thing, and this is one of the most valuable aspects of the Agile (big A) approach – all of the methods require of the team that they re-evaluate the business case at the end of every iteration. I note the “waste” figure in 2009 is US$22BN – in 1994 it was US$73BN (in 1994 $$$). “Failure” numbers have gone up but the waste figure has gone down, which seems to me to indicate that more projects are being stopped earlier – which is a good thing.
I don’t think we have degraded as an industry, but I do think we’re better able to stop bad work early.
Offshoring/outsourcing results in approx 30% outright failure (contract cancellation) AFTER spending a year or more trying to deliver products. This in my mind is truly waste.
Please don’t blame “Agile” for organisations that are actually implementing Fragile (or Tragile) development approaches. Any methodology or approach implemented badly – without regard for the disciplines and dynamics required to deliver working products – will result in waste rather than business value delivery. For me Agile approaches expose this waste as early as possible and enables the project team to change direction or stop the work, building on the good practices from Iterative/Incremental development (eg RUP).
Tragile/Fragile development repeats all the problems and mistakes we’ve had as an industry since the 1960’s with the same predictable results. Just calling it “Agile” doesn’t make the people or processes any better, in the same way as many organisations said they adopted RUP or RAD or iterative development, or any one of the good practices that have appeared over the last 30 years; what they really did was change the language but continue making the same mistakes, the same thing is happening today with Agile, and will happen in the future when a new brand appears.
It is easy to hide dysfunction in sequential development projects until right at the end, and often the organisation has too much invested by that stage to admit the problem, thus delivering poorly fitting systems that don’t meet the business needs.
Are we actually going backwards? I’m not so sure – I’m reminded of Tom DeMarco‘s talk at SDC a few years ago: the reason we are having trouble building systems today is that all the easy ones have been built – what we deal with today are more complex integrations, more complex technologies and more demanding customers.
If I consider just one organisation I’ve worked with lately they used to have COBOL applications running on a single mainframe platform using green-screen technology to record and report on the state of farm animals in New Zealand. A farmer filled in paper forms with milk production and breeding records which were hand punched into the mainframe and the farmer then received a quarterly report of their herd status. Today each cow has a RFID tag, as the cow is milked the exact quantity of product is recorded by the milking machine and transmitted via wifi to the farm central server, and synced to the handheld device the farmer carries with him/her from which they can scan an animal and immediately see the full veterinary history, calving records, genealogy and milk production records enabling them to make better informed decisions. (I’m not even going to try and describe the breeding events and the details of bull productivity which they track). They credit the changes in technology with a significant increase in milk productivity per animal across the NZ dairy herd – and an increase in wealth for farmers as a result. The complexity of technologies and interfaces is orders of magnitude more complex than the mainframe systems yet their IT budget as a proportion of revenue has remained fairly static, and measured in business value delivered they are providing significant return on the money spent.
Underlying all this is the important recognition that software is built by people – people effectively working together with a focus on the customer’s needs will produce good results; people not collaborating and not communicating effectively will produce bad results. Irrespective of the brand or type of process you apply it’s the people and relationships that make for effective software development.
I look forward to hearing from others.
Posted by Shane Hastie