I've been doing a lot of research lately on project failure and I have noticed some interesting patterns regarding denial and the fear of admitting failure or mistakes.
Since 1994, the Standish Group has been conducting bi-annual studies of IT projects and the success and failure rates for those projects. Dubbed the Chaos Studies, the research is widely quoted as the industry standard (though recently there is some criticism of the data). Since the inception of these Chaos Studies, the reported statistics for projects that fail, are challenged, or succeed have been roughly the same (see summary below). The cost of these failed projects and the amount of waste represented is staggering but that is not the point of this post.
The point of this post is not project failures (we'll tackle that later), it is fear and denial. I have been out talking to people about troubled projects and referencing these Chaos Studies. I have found that very few IT Leaders, CIOs, and IT project managers are even aware of these studies and failure statistics. When asked they do acknowledge that there are a high rate of failures but don't pay attention to industry studies on it nor do they think of it as a call to action as I do. To me this seems the rough equivalent of a pitching coach not knowing the ERA for any of his pitchers or a life insurance salesperson not being aware of actuarial tables.
But it also seems to me that there is another factor at work and that is a professional disdain for telling the truth (aka denial). As a program manager, I have seen many resumes for project managers. The vast majority of those resumes state that all the projects were delivered "on time on budget" as if it were a mantra. If only roughly 1 in 3 projects are considered successful, there must be someone responsible for the other 2 in 3 projects that were not successful. Somebody is not quite owning up to those problem projects.
A recent LinkedIn discussion post touched on a closely related topic - "Do we really think someone will declare their project is in trouble". The spirited discussion revealed some interesting attitudes about fessing up to a problem project situation. Some of the opinions stated were that it's a professional/ethical responsibility to declare you are in trouble; it is a sign of a good project manager to own up and ask for help; we learn the most from our failures; anyone who doesn't own up should be tarred and feathered, etc. One common thread in many of the responses was a need for an external QA function to provide an external view on the health of the project. That is, we need someone other than the PM to tell us what is really going on.
Here are my personal opinions on why people don't admit their failures or reach out when in they are in trouble. I believe that they are afraid; afraid that people will think less of them or that they are incompetent. I also think that some people, project managers in particular, are afraid to ask for help because it makes them look weak.
While the Chaos Studies may be skewed in their reporting of failure, I think most of us would readily admit that there is a relatively high rate of failure on IT projects. We need to have less denial about the start of our industry, more honesty about the state of our own projects, and turn this around so that we see failure as part of the learning process. We need to not only acknowledge our own failed initiatives, but harvest them for the lessons they can provide.
Tina Seelig wrote about failure as a learning opportunity in her 2009 post, You can't spell failure or success without U. She talks about how she requires students in her class to write a failure resume showcasing all the mistakes they have made. She sees this as a valuable reminder to extract all the lessons from failure and to see failure as a key part of the learning process. As she has noted, if you want more successes, you have to tolerate more failures.
Here are what I see as the lessons for project managers:
- The truth always comes out and so it is best for you to share it. You will be more favorably regarded by telling the truth rather than denying problems or reaching out for help. It may take a while, but the truth eventually comes out.
- Reach out for help when you need it. Asking for help is not only responsible, it is also good and appropriate communications and expectations management. Besides, you stand a much better chance of heading off major problems before they grow if you are willing to admit you need help and seek it.
- When hiring project managers, watch out for those with a spotless record. Be suspicious of those who say 'on time on budget' as if it were a mantra or a secret pass phrase. Ask people about their failures and see what they are willing to share.
- Harvest every project for the lessons to be learned. A project post-mortem should be conducted for every project, especially those projects that are canceled mid-stream. Also, PMs should maintain a journal of every success, failure, and lesson learned through the execution of the project. Ideally this is something that is done using a few minutes each day or once a week.
- Expect failure as a requirement for success. If you and the people around you are not failing with regularity, you are not trying hard enough or taking enough risks.
IT Project success rates aren't going to improve until we start telling the truth about where we are and what is causing the failures. We need to extract the lessons from our failed projects and use them to continually improve our project success rates.