CHAPTER 1

Why Do 70 Percent of Projects Fail?

We should be careful to get out of an experience only the wisdom that is init—and stop there; lest we be the cat that sits down on a hot stove-lid.She will never sit down on a hot stove-lid again—and that is well; but shewill also never sit down on a cold one anymore.

—Mark Twain, Following the Equator, 1897

Why Projects Fail

If project management is such a good idea, why do 70 percent of all projectsfail, including those led by experienced and capable project managers? Why doesit seem to be so difficult to get projects done within the Triple Constraints oftime, cost, and performance—or, in layperson's language, on time, onbudget, and to spec?

Here are a few instructive examples of some of the more recent spectacularfailures in project management:

* In 2006, a $400 million purchasing system for Ford Motor Company was simplyabandoned.

* Software errors in a U.K. Inland Revenue system resulted in a $3.45 billiontax-credit overpayment.

* The infamous automated baggage system at Denver International Airport burnedthrough $250 million before being abandoned as un-workable.

* The U.S. Department of Defense's $6 billion Kinetic Energy Interceptor programwas terminated in 2009 after it was determined that it would not achieve itsgoals.

That's not all. Let's look at some numbers on project performance, compiled bythe Standish Group. This organization has tracked project performance since1994. Every two years, the Standish Group issues the CHAOS Report, whichanalyzes projects primarily in the software area. In the 2009 CHAOS Report, theyreported these abysmal numbers:

* 32 percent of projects were delivered on time, on budget, and with therequired features and functions.

* 44 percent were finished either late, over budget, or only partiallycompleted.

* 24 percent failed altogether, and they were canceled or abandoned.

There's good news and bad news here. The good news is that in 1994, when theStandish Group began tracking data, only 16 percent of projects succeeded inmeeting the Triple Constraints (on time, on budget, to spec). On the other hand,the 2009 report shows that there's been a downtick in success (34 percent to 32percent) and a significant uptick in failure (from a low of 15 percent to 24percent).

For challenged projects, those that succeed in some elements and fail in others,the good news is that average budget overrun has dropped from 180 percent toonly 43 percent. On the less positive side, time overruns have gone up 30percent, and the percentage of features that have made it into the final producthas dropped from 67 percent to 52 percent.

During this time, nearly 260,000 project managers earned the prestigious ProjectManagement Professional (PMP) designation from the Project Management Institute(PMI). But the track record of improved project performance is lackluster atbest.

What's going on?

A significant amount of study and reporting going back several decades has shedlight on some of the reasons for these failures:

* The 1998 Bull survey, conducted by the French computer company Bull,identified the major causes of information technology (IT) project failure as abreakdown of communications, a lack of planning, and poor quality control.

* KPMG Canada, in 1997, identified the core project failure issues as poorplanning, weak business case, and a lack of top management involvement andsupport.

* The Standish Group's 1995 CHAOS Report named incomplete requirements and lackof user involvement as reasons for project failure.

* The OASIG Study, published in 1995 by a U.K. group studying organizationalaspects of information technology, cited lack of attention to the human andorganizational aspects, poor project management, and poor articulation of userrequirements as reasons why projects failed.

But poor planning, weak business case, and inattention to human andorganizational aspects aren't causes; rather, they are symptomsof a much large systemic shortcoming. Treating the symptoms isn't the same astreating the underlying conditions. We know some of the root causes. People withpoor interpersonal or team leadership skills create friction, as well asstakeholder conflict, in the project environment. Friction then increasesinefficiency and waste. The size and complexity of an organization increases itsmoment of inertia, and getting anything to move takes enormous effort. Peoplecome and go, missions mutate, information goes missing, and ultimately entropyincreases—we tend to move from order toward chaos.

Things fall apart. It's been said that there are only two reasons for projectfailure:

1. Things that nobody thought of or prepared for

2. Things that everybody thought of, but nobody did anything about

If you think about it, these reasons alone cover almost every potentialincident. How often have you experienced project problems because a couple ofthe people working on your project were suddenly pulled off halfway through? Howabout a major change ordered in one or more of the Triple Constraints when theproject is three-quarters completed? Perhaps there's some recurrent problem inthe project environment that manages to happen every single time. Things takelonger than you expected. Not everybody is really on board. There's always alayer of technical complexity no one expected. Stakeholders don't really knowwhat they want, or they expect you to figure it out magically. All of theseproblems have the same result: a mess.

But do you account for these situations in your project planning? For a fewoutstanding project managers, the answer is at least a partial yes. For most ofus, the answer rests somewhere between seldom and never.

Four Essential Project Questions

If you take the list of reasons from the studies mentioned previously, you canboil them down into the following four (often unasked) questions:

1. Why are we doing this? (Business case)

2. Who has an interest in what we're doing, and what do they each want and need?(Human and organizational aspects)

3. What do we have to do, and how are we going to do it? (Project management,including planning and quality control)

4. Who needs to be involved, and in what way? (Top management and userinvolvement and support)

The official standards of professional project management are designed to makesure these issues get appropriate consideration. But these considerations arequite obvious—it shouldn't take a PMP to grasp these concepts. Why, then,given the amount of effort, knowledge, and resources, is the situation in someways getting worse?

The Operational Art of Project Management

The Project Management Institute (PMI) defines project management as "theapplication of knowledge, skills, tools, and techniques to project activities tomeet project requirements." That's fine, but it's not nearly enough. Toolsmatter, of course, but a hammer and saw don't make someone a carpenter. Nor doesmastery of technical skills alone ensure success.

To us, the key issue is thinking well—a focus on practicalcreativity that combines brainstorming, operational analysis, and planning tohelp you solve problems, find opportunities, and gain insights into any project.

Thinking well is a broad topic that includes many issues of interest to projectmanagers, for example:

* Thinking outside the box (or for Triple Constraints–oriented projectmanagers, thinking outside the triangle)

* Thinking clearly about the circumstances and environment in which our projecttakes place

* Thinking honestly about risks and opportunities

* Thinking about our own biases and blind spots so we can minimize their harmfuleffects

What project managers learn (some of us do so the hard way) is that the self-imposed constraints, assumptions, and opinions we and other stakeholders bringto the project manifest themselves subliminally in a variety of ways that toooften hinder project performance.

Fundamentally, project management is an operational art; it's the linkbetween strategy and tactics. Project management is the operational art thatapplies the goals of the project to the tasks we perform. Just as there's anoperational art to getting an army (equivalent to a small city) to move, there'san operational art to building a skyscraper or leading a large IT project.

The Seven Dimensions of Project Management

Projects differ from operational work because projects end. Bydefinition, they are "temporary and unique." Projects take place underconstraints. Projects have different levels of complexity and different levelsof uncertainty. Project managers live in a bounded, finite universe ruled byscarcity and governed by the Triple Constraints of time, cost, and performance,as shown in Figure 1-1.

The Triple Constraints themselves array in a hierarchy of driver, middle, andweak constraints. The driver is the leg of the Triple Constraints that drivesthe project. If you're rushing to beat the clock, time is the driver. If there'sonly so much money and not a penny more, cost is the driver. If getting itexactly right is essential, performance is the driver. The weak constraint, onthe other hand, is not necessarily the least important constraint, but it isalways the most flexible. That flexibility is where many creative solutions tendto live, so knowing not only which constraint is weak but where it is weak is ahuge opportunity for any project manager. The middle is, well, in the middle.There may be exploitable flexibility, but not as much.

Each of the six possible combinations of time, cost, and performance forms aseparate dimension of project management (Dobson/Feickert, 2007) and providesits own set of challenges and opportunities.

A Man, A Plan, A Gantt—Project Management

When the only tool you have is a hammer, all problems start to look like nails.The tendency to overuse the familiar tool (as opposed to the correct one) ispart of our general proneness to prefer the familiar to the strange, the knownto the unknown.

That line of thinking is a mistake. A creative project manager must accept thatnot everything is, can be, or should be familiar, known, or controllable. Tounderstand how formal project management can mislead its modern practitioners, abrief history of its origins is necessary. Project management grew out of aproduction and engineering environment. In the process, rules, more rules, andeven more rules were created. Project management is not production, however; itis the application of a standard production process to a unique and creativeevent. We tend to manage the creativity out of projects that are by definitionunique and creative. That is the root cause of many project problems.

The project management profession has focused, reasonably enough, on performanceimprovement. To that end, the act of project management has been deconstructed,sliced and diced, and studied from a multitude of vantage points and technicalspecialties. The result has been a consistent effort to demystify projectmanagement by documenting centralized processes, to apply the rubric of"scientific management" so that projects become repeatable and controllable.It's a worthy goal. We question whether it is a realistic one.

Today, the center of gravity of the project management world is the ProjectManagement Institute. In 2008, it reported a membership of 260,000 practitionersoperating in 171 countries. PMI's standard reference, A Guide to the ProjectManagement Body of Knowledge (popularly known as the PMBOK Guide) isthe bible for people hoping to earn the designation of Project ManagementProfessional by passing a challenging examination. This guide is an intellectualheir to the theory of scientific management.

In the 1880s and 1890s, Frederick W. Taylor pioneered the systematic analysis ofworkflows, hypothesizing that traditions and rules of thumb were insufficient tomanage the radical new technologies brought on by the industrial age. Thatexamination was scientific in that its conclusions were developed throughcareful study and analysis, not based on the whim or preference of any specificworker. This process of scientific management, Taylor believed, would naturallyresult in increased efficiency and productivity, combined with lower waste.

Taylorism and modern project management were joined at birth. Projects, ofcourse, are as old as human civilization, but the story of project management asa formal discipline begins with Henry Lawrence Gantt (1861–1919), anAmerican mechanical engineer and management consultant. Gantt, famous for theeponymous Gantt chart, was Frederick W. Taylor's college roommate and laterworked with Taylor to apply scientific management to the steel industry.

Besides his chart, Henry Gantt is famous for two other accomplishments. He iscredited as the originator of the idea of linking management bonuses to how wellthe managers have taught their employees to improve performance, and heestablished a formal model for industrial efficiency.

You Say You Want an Industrial Revolution ...

Both Taylor and Gantt were children of the Industrial Revolution, atransformational moment in human history. Old ideas about work crumbled underthe impact of new technology, and processes had to change. Unlike agriculture,in which a farmer can do everything right and still have a crop fall victim to anatural disaster like an early winter or a prolonged drought, the IndustrialRevolution held out the hope of certainty. Machines, at least in theory, arepredictable, repeatable, and efficient. If only workers could learn to be morelike machines, we would shortly all live in a brave new world of controlled andmanaged happiness.

It's absolutely true that scientific management tamed the new technology of theIndustrial Revolution, created new and valuable ideas about productivity andefficiency, and made the world a better place. It's equally true that there aredownsides and costs associated with the new goals of the modern age. From AldousHuxley's Brave New World to George Orwell's 1984, from FritzLang's Metropolis to Charlie Chaplin's Modern Times, scientificmanagement and the Industrial Revolution have been portrayed as dehumanizingworkers and imposing tyrannical control over the smallest details of humanbehavior.

There's plenty of truth to go around.

In the field of risk management, we talk about "secondary risk," the new riskinadvertently created by your attempt to mitigate the original risk. It's wellknown that solutions often create new problems.

Mechanical metaphors can only take us so far. Even an infinite set ofchecklists, databases, and Intranet sites filled with updated Microsoft Projector Primavera files and a fully staffed Project Management Office (PMO) filledwith certified PMPs have clearly not solved the problem of failed projects andin some ways make it worse.

At one time it was possible for an educated person to learn almost everythingthere was to learn. That hasn't been the case since the late Renaissance period,and as a result people have specialized. Specialization allows people to developgreat expertise, but it complicates creative cross-border thinking and createsits own special cognitive bias, known as déformation professionnelle.That's when people look at every problem through their own narrow lens,forgetting that any other points of view exist.

Adaptability

The conflict between Theory X and Theory Y, between chains of command and thematrix organization, and between efficiency and teams has kept the authors andpublishers of management books in business for generations, for which everyoneassociated with Creative Project Management is deeply grateful. Like allattempts to perform balancing acts on slippery slopes, we must make continualadjustments. Sometimes these adjustments are a function of attitude andtemperament; sometimes they are a function of a shifting environment or thecharacteristics and constraints of the project.

It is well known that project management must be scaled, but it must also bestretched.

The systems and processes needed to manage the construction of a new aircraftcarrier would be gross overkill if they were used to build a patio in one'sbackyard. Both projects are temporary and unique. Both can be broken down intopackages of work. Both have measurable end states. But that's about all theyhave in common. Scale affects costs, and it limits your choices.

Projects in creative or design fields often require agile approaches. When theborder between tasks blurs into a transition, when the work iterates instead ofprogresses, and when collaboration crosses boundaries at will, projectmanagement must also be stretched. The systems management virtues of formalmethodology weaken, and uncertainty replaces it.

This outcome makes many traditional project managers extremely uncomfortable.

The Mental Effects of Uncertainty

The American National Standards Institute (ANSI) accredits PMI's PMBOKGuide as a standard for project management practice. That means its mission,in some ways, is to finish what Henry Gantt started: to ensure that thecharacteristics and performance of processes are consistent and that people usethe same definitions and terms. It is fully compatible with Gantt's concept thatindustrial efficiency can only be produced by the application of scientificanalysis to all aspects of the work in progress.

Like Henry Gantt, PMI appears to believe that the essential goal and aim ofproject management is to eliminate chance and accidents.

We believe that not only is that goal impossible but it is also not necessarilyeven a good idea.

The Chaos Paradigm

The age of machines has pointed the way toward a utopia of predictability, butin the age of computers and biotechnology, chaos seems much more the norm. Whileit's a good idea to tame what can usefully and practically be tamed, most of theproject world lives where the wild things are. Chance and accidents have givenus penicillin, vulcanized rubber, and that most essential tool for modernproject managers, the Post-it Note. (You can have our copies of MicrosoftProject, but you'll have to pry the Post-it Notes out of our cold, deadfingers.)

(Continues...)