Über die Autorin bzw. den Autor

Patricia A. Patrizi is chair of the Evaluation Roundtable and principal of Patrizi Associates, Philadelphia.

Michael Quinn Patton is founder and director of Utilization-Focused Evaluation, and author of the book by that name. He is a former president of the American Evaluation Association.

Auszug. © Genehmigter Nachdruck. Alle Rechte vorbehalten.

Evaluating Strategy

John Wiley & Sons

Chapter One

Michael Quinn Patton, Patricia A. Patrizi

Abstract

Strategic thinking and planning have long been the focus of management training and organizational development, but strategy is a new unit of analysis for evaluation. The authors examine the increasing attention given to being strategic in the private sector, in government, in philanthropy, and in the not-for-profit sectors. To respond and adapt to concerns about the implementation and impacts of strategy, strategy needs to be evaluated. The authors look at what strategy is, offer a framework for evaluating strategy, consider issues in strategic evaluation, and examine opportunities for strategic learning based on evaluating strategy.

Thinking and Evaluating Strategically

Unless a variety of opinions are laid before us, we have no opportunity of selection, but are bound of necessity to adopt the particular view which may have been brought forward

Herodotus, Greek historian, 5th century B.C.

The contemporary word strategy derives from the ancient Greek word strategos, which meant to think like a general. The term originated, then, as a reference to military strategy, but in ancient Greece military leaders were often also territorial governors. Strategoi, then, were politicians as well as generals.

Strategy is a new unit of analysis for evaluation. The Encyclopedia of Evaluation (Mathison, 2005) has an entry on "strategic planning" but nothing on strategy as an evaluand or on evaluating strategy. Traditionally, evaluation has focused on projects and programs. Organizational development makes the organization the unit of analysis for assessing organizational effectiveness, usually focused on mission fulfillment. Management, in contrast, often focuses on strategy as the defining determinant of effectiveness. The language of strategy permeates senior management initiatives in government, philanthropy, the private sector, and the not-for-profit world. Being sensitive and responsive to primary intended users of evaluation has become a hallmark of effective and useful practice. In that sense, being sensitive and responsive to the language and concepts used by primary intended users is a strategy for enhancing the relevance and utility of evaluations. We came to focus on the question of what it means to evaluate strategy, because we observed that our senior management clients in all sectors were talking not about theories of change or logic models, but about being strategic: Strategic thinking. Strategic planning. Strategic results. Being strategic. Strategy execution. Adapting strategically. And, yes, evaluating strategy. In this regard, we are working in the tradition of that eminent evaluation pioneer, Sir Winston Churchill (1874–1965), who is reputed to have observed: "However beautiful the strategy, you should occasionally look at the results."

One of the reasons we think that evaluators have been slow to focus on evaluating strategy is that strategy is closely associated with planning, strategic planning, and as evaluators we don't do planning, we do evaluation. But what's been happening in the world of management is a movement away from a focus on strategy as equivalent to planning, something we'll have more to say about later. For now, we want to emphasize that evaluating strategy is not about evaluating strategic planning, or even strategic plans. It's about evaluating strategy itself. And that, we shall argue, makes all the difference.

We begin, then, with a brief overview of how the language of strategy has come to permeate management thinking and writings about organizational effectiveness. We'll then examine some of the diverse definitions of strategy and what it means to be strategic. Then we'll review what we consider to be a particularly useful framework for evaluating strategy. Finally, we'll consider the implications for evaluation of treating strategy as a distinct evaluand, that is, a specific unit of analysis and target for evaluation.

Strategy as a Focus

We want to begin with the larger context by giving some sense of how strategy is talked about by organizational leaders, board members, academics, management consultants, best-selling authors of management books, and influential professional development trainers. As an interdisciplinary and transdisciplinary field, it is important for evaluation to keep abreast of developments in the larger world. What are leaders talking about? How do they conceptualize the challenges they face and the value-added they bring to their organizations? What do they worry about? What are they seeking to improve? That which concerns them should concern us. At least that's the premise under which we're working.

Strategic management has long been a private-sector focus and there are lots of books about strategic management (e.g., David, 2008; Hill & Jones, 2009; Hitt, Ireland, & Hoskisson, 2010; Pearce & Robinson, 2010). You'll also find a plethora of books on related themes like strategic thinking (Haines, 2007; Sloan, 2006) and strategic planning, including books targeted to the government and not-for-profit sectors (e.g., Bryson, 2004; La Piana, 2008). But we found nothing on strategic evaluation.

There is, however, a recognized field of strategy consulting. This is one of the specializations monitored in the annual management consulting recruitment channel report. The 2010 report found that strategy consulting is especially vulnerable to economic conditions.

Strategy consulting, often the quickest to be hit in a downturn, can also be amongst the last to recover during an upturn. In many respects it's the most discretionary type of consulting spend there is—and this has spillover effects on the hiring trends within strategy consulting. Given this, it is therefore heartening for the industry as a whole to see a general consensus that strategy consulting hiring will gather pace this year—one of the strongest indicators that as an industry we perceive the worst to be very much behind us. (TopConsultant, 2010, p. 10)

One approach to strategy consulting is strategic performance management. For example, Rick Johnson, a private-sector management consultant who founded CEO Strategist LLC, markets himself as a "leadership strategist" and asserts that: "Success during the most significant economic challenge since the depression, is Strategic Performance Management" (Johnson, 2008, p. 1). He argues that the big mistake of the past has been too much focus on performance measurement rather than on strategic performance management, which he presents as a "platform for the effective management of individuals and teams in order to achieve high levels of organizational success ... Strategic performance management is a holistic process, bringing together many of the elements which go to make up the successful practice of people management, including in particular—learning and development" (p. 1). A related approach, Strategy-Driven Execution Management, is a proprietary model based on monitoring the implementation and outcomes of strategy (Keyne Insight, 2010).

Government initiatives also use the language of strategy. For example, the federal Office of Management and Budget created...