Powerful Beyond Measure? Measuring Complex Systemic Change in Collaborative Settings

Adinda Van Hemelrijck

Introduction

Over the past two decades, there has been considerable debate regarding the effectiveness of aid and its contributions to development. The demand for rigorous performance and impact measurement for making evidence-based arguments has increased significantly. As a result, the debate has revivified the old paradigm war over methods (Patton, 2008:420-421). Recognizing the depth and importance of the methodological debate, perhaps it would be more apposite and fruitful to move beyond the dispute and make better use of all worldviews in an integrated, flexible, and responsive manner. A comprehensive approach is needed for addressing the aid effectiveness challenge: one that produces different kinds of evidence over a longer period of time, sheds light on the development issues from different perspectives, and builds arguments that can convince development actors to commit to their respective roles more responsibly (Fowler, 2007, 2008; Guba & Lincoln, 1989; Khagram et al., 2009; Mertens, 2009; Patton, 2008, Servaes, 2007).

In this chapter, I argue that in order to provoke the right conversations among key players in the development process about what works, what doesn't, why, and what are plausible alternatives; the data must be meaningful, useful, and respond to their information and learning needs. In other words, the approach should be primarily focused on evidence-based learning. At Oxfam America, this proposition led to a rights-based and systems-thinking approach to impact measurement and learning, based on the understanding that fighting poverty and injustice calls for fundamental systemic changes. To measure these changes, a methodological fusion is required to capture the attendant complexities and present them in a way that can meet and influence stakeholders' different worldviews. This is where communication for sustainable social change and development (CSSC&D) comes into play: it provides a conceptual and methodological framework for encouraging stakeholders to engage meaningfully in the development process from their own perspectives, and to be accountable to learn and act upon, or behave in accordance with, what they have learned in dialogue and collaboration with others.

This chapter explores Oxfam America's approach to program design and impact measurement and learning from a CSSC&D perspective, which in essence is about the means and ends of empowerment. Impact measurement can be both meaningful and feasible; this chapter attempts to unpack the key aspects that define its success, and elaborates briefly on the methodological implications from a culture and sustainability perspective. The approach is illustrated with a practical example: Oxfam America's program on smallholders' productive water rights in Ethiopia. By comparing theory with practice, underlying principles and assumptions are revealed and important challenges identified. Hopefully this will foster a dialogue between academics and professionals that can help Oxfam and other international agencies find more sophisticated and knowledge-based answers.

A rights-based and systemic approach to development

Context and background

At the end of the twentieth century, more than 50 percent of the six billion people on earth earned less than $2 a day, 30 percent less than $1 a day. The statistics triggered a worldwide debate over the effectiveness of aid: it didn't seem to work as expected, didn't reach the poor, and turned out to carry many negative side effects (François et al., 2006).

In the post-Soviet international environment, a more persistent call emerged for broader democracy, good governance, ownership and partnership, donor coordination, demand-driven aid, and a greater focus on human development. In the 1990s, a new development paradigm and a new system of global governance based on international law began to emerge. While in the 1970s and 1980s, the focus was predominantly on technological modernization, economic growth, and structural adjustment – with the interdependency and basic needs approaches as alternative critiques – poverty reduction, sustainable development, and human rights now rose to the top of the international development agenda (Coolsaet, 2006; Develtere, 2005; Fowler, 2003; Robb, 2004).

The essential beliefs or assumptions underlying this new paradigm or model were that:

(a) States and markets are each other's necessary complements, and well-functioning institutions are necessary for sustainable growth and development (Bourguignon, 2004; Roche, 2012; Wallace et al., 2007).

(b) Economies grow and develop best and more sustainably when all people equally participate in, benefit from, and take responsibility for growth (Edwards, 2011; Edwards & Hulme, 1992; Rowlands & Eade, 2003).

(c) Poverty is more than simply insufficient income, but is also the inability to choose, attain, and sustain a decent living standard. Poverty is complex and systemic, caused by multilevel and multidimensional exclusion and marginalization (Green, 2008; Kakwani & Silber, 2007; Kanbur & Lustig, 2001; Sen, 1999).

(d) All people are responsible citizens who have the right to development that includes dignity, well-being, equality, freedom, and self-determination. People have fundamental and universal human rights, which are codified in international laws and treaties and should be enshrined in moral and ethical norms that govern human societies (Chambers, 2008a; Cornwall, 20 04; Green, 2008; Kakwani & Silber, 2007; Kanbur & Lustig, 2001; Sen, 1999; Toye, 2011).

(e) Poor individuals, families, households, and communities are meant not just to benefit from but also to be enabled by development to live and act as responsible and dignified citizens (Gaventa, 2002; Green, 2008; Johnson, 2003; Offenheiser & Holcombe, 2003).

(f) Understanding the local sociocultural and political-economic context is imperative for effectively and sustainably addressing the key determinants of poverty (Chambers, 2008a, 2009, 2011a, 2011b; Groves & Hinton, 2004; Wallace et al., 2007; Woolcock, 2009).

(g) Local realities are influenced and shaped by wider and global systems, though local dynamics also influence the broader systems. Development is shaped by trends and processes occurring at multiple interacting levels, manipulated by global policies (Jupp et al., 2010; Mayoux, 2007; Mayoux & Chambers, 2005).

Based on these assumptions, it is argued that sustainable and human development should entail a wide variety of strategies that focus simultaneously on the equal distribution of economic growth and opportunities, on gender justice and human rights, environmental protection, strengthening local institutions, and democratic governance and market inclusion at multiple levels. This requires new types of partnerships (including private-public) and the active engagement of multiple legitimate stakeholders who weigh the objectives and observe the relationships from different perspectives (Green, 2008; Khagram et al., 2009). Those who are living in poverty should be viewed as the primary constituents, and the primary agents, of development. The focus of aid should be on creating opportunities, expanding choices, and enlarging freedoms through building capacities, rather than increasing income through economic growth and providing services in response to poor people's basic needs (Chambers, 2009, 2011b; Lucas & Longhurst, 2010; Roche, 2012). Interventions should be designed based on sound multidisciplinary and context-specific analyses, recognizing the crucial role of local culture, networks, and institutions to cope with market failures, and mobilizing communities as the primary stakeholders of development. Therefore, interventions need to be better aligned and selectively more tuned to the objectives of their primary stakeholders and constituents (Chambers, 2008a; Edwards, 2011; Green, 2008; Kanbur & Lustig, 2001; Sen, 1999).

In today's neoliberal context and after nearly five years of triple crisis (e.g., food, energy, and financial), it appears as if the new concept and paradigm of development is interminably in flux. While poverty continues growing, development issues such as growth, well-being, rights, and equality are becoming more complex. Local and global perspectives are making room for multidimensional, multilevel, and multiscalar systems-thinking approaches that build on a more dynamic and complex interrelational view of poverty and related development issues. This trend challenges traditional ways of measuring and explaining aid and development. Alternative approaches to impact measurement and learning are needed that can help key stakeholders better understand what it means to engage in development in a world characterized by multiple crises and power shifts culminating in higher degrees of uncertainty and unpredictability (Bawden, 2010; Chambers, 2011a, 2011b; Groves & Hinton, 2004; Khagram et al., 2009).

Development as rights and empowerment

By enforcing the application of the rights principles enshrined in international agreements, conventions, and laws, sustainable and human development practice became increasingly more rights-oriented (Cornwall & Nyamu-Musembi, 2005; Fowler, 2003; Johnson, 2003; Servaes, 2005; Uvin, 2004).

The rights-based approach to development generally builds on the belief that poverty and injustice have to be considered essentially as rights issues that are convoluted by the multilevel nature of rights violations and moral obligations. The problems people face are not the "natural" human conditions; they are the consequences of complex sociopolitical relationships, structures, and local-to-global market mechanisms that sustain and (re)generate inequality and injustice. This cannot be fixed by short-term interventions or the scaling-up of technical solutions. The symptoms can be fought temporarily (as famine is by food aid, or the lack of water by digging wells), but its root causes require more fundamental and sustainable systemic changes (Cornwall, 2004; Fowler, 2008; Green, 2008).

From a rights perspective, development is the process whereby people become increasingly aware of their rights, learn from their own and others' experiences, appreciate the global processes of change engulfing them, grow in their ability to adapt and cope with change, face challenges with dignity and perseverance, learn how to harness opportunities in a respectful and sustainable manner, and acquire the ability to access and influence institutions to claim their rights, and hold development actors accountable. Development can therefore be understood as freedom or empowerment: the ability of people to influence the wider system and take control of their own lives (Annan, 2005; Fowler, 2003; Khan, 2004; LEAD, 2008; United Nations, 2003, 2004; Van Hemelrijck, 2009).

The human rights approach adds an ethical and moral dimension to development based on human dignity. It presents a powerful proposition for applying an expanded, more relational notion of accountability, building on the principle that all development actors have the duty to respect and protect the individual and collective rights of legitimate rights-holders, who are equally obligated to conform to law and respect others' rights. Backed by international law, a strong legal-normative basis is provided for poor people to make claims on their development status, and hold to account duty-bearers such as state governments, private sector, civil society, and other development actors (including Oxfam) for enhancing their rights. This can be challenging when complex power dynamics are at play and people are threatened and have limited or no access to adequate legal services and other local institutions to protect them. Broader support from within and outside their societies is needed to demand accountability and justice: local and global citizens, public leaders, media, and other societal and international actors have the moral obligation to demand fairness, equality, democratic governance, and respect for human rights. They, too, are part of the rights and relational accountability framework (Eyben, 2008; Gready, 2008; Green, 2008; Nyamu-Musembi, 2005; Offenheiser & Holcombe, 2003).

Communication for development in Oxfam's rights-based programs

From a rights perspective, participation is seen as more than just an instrument for effective grassroots intervention. It is concerned with "who is in control" and is meant to broaden people's power and influence in the priority-setting, creation, and analysis of systemic solutions. Grassroots-level participation can be scaled and people's influence broadened through social mobilization, coalition-building, and advocacy. In this rights-based approach, multiple spaces are created for poor people to meaningfully engage by applying communication strategies and methodologies that are commonly delineated as CSSC&D. This field of work is generally defined as a well-planned, evidence-based, long-term, and participatory process that seeks to empower poor people to shape their own lives, culture, and development; influence attitudes and opinions among citizens; and promote democratization and participation at local, national, and global levels (Bakewell et al., 2005; Gaventa, 2006; Gaventa & Cornwall, 2006; Hickey & Mohan, 2004; McKee & Aghi, 2000; Puddephatt et al., 2009).

Using such a rights and empowerment perspective, Oxfam America has developed a programming approach that is similar to CSSC&D. Different, however, from more standard models is its purposeful design to deliver impact that is defined and measured as "empowerment" in the sense of (LEAD, 2008; Van Hemelrijck, 2009, 2010):

a significant and sustainable change in power relations that enables excluded and marginalized people to realize their rights to access and manage the resources, services and knowledge they need for strengthening their livelihoods, improving their well-being, and influencing and holding accountable the institutions that affect their lives.

Only long-term collaborative efforts persistently focused on a specific rights issue and simultaneously working at grassroots, civil society, and various governance levels can hope to realize such impact. This is why Oxfam America and its partners develop ten-fifteen years programs that build upon a sophisticated knowledge of the local context and drive on a broader coalition for deploying strategic interventions synergistically geared to achieve a common impact goal. These interventions typically include grassroots community mobilization, market inclusion and private sector engagement, movement building, policy and advocacy work, and rights monitoring and impact research.