TRENDS

The national investment in composition instruction remains huge, despite recent budget cuts throughout the educational system. At the high-school level, we can count 358,136 students taking the advanced placement examination in English language and composition (College Board 2013, Table 9), the better-prepared tip of a population concerned with entry-level college writing. The National Center for Education Statistics (2012; Hussar and Bailey 2008, Fig. C) reports that total enrollment in degree-granting institutions increased 23 percent from 1992 to 2006. Between 2006 and 2017, a period of only eleven years, enrollment is projected to top at least 19.4 million students, almost all of whom will be enrolled at some level in college writing courses.

Such growth is accompanied by new challenges. Citing a 300 percent rise in average tuition during the past three decades, an average bachelor's degree debt of more than $29,400, and a graduation rate of only 58 percent for a four-year degree within six years, the Department of Education has launched federal initiatives to ensure quality in postsecondary institutions (White House 2013). Of special interest is the emerging Postsecondary Institution Ratings System (PIRS) that will be used to produce individual college scorecards linked to student financial-aid levels. Students attending colleges with higher ratings could, for instance, be eligible for larger Pell Grants and favorable rates on student loans. The opposite would be true for students at colleges with lower ratings (National Association of Student Financial Aid Administrators 2014).

In light of such developments, it is both reasonable and responsible for administrators and public officials to inquire into the effectiveness of the writing instruction programs in which students are enrolled, particularly at two crucial transfer points: between high school and college, and between graduation and career development. For over fifty years, composition researchers and assessment specialists have been attempting to provide reliable evidence — beyond the felt experience of teachers — that composition instruction leads to valuable categories of student improvement, including evidence that students are better writers. But the results have been infrequently laudable, often mixed, and sometimes disappointing.

WRITING PROGRAM ASSESSMENT: A DISCOURAGING HISTORY

Among the many reasons for this complex situation, perhaps the most important is the difficulty of defining writing and the objectives of writing instruction at the transfer points we have described. This complexity is deepened when assessment is understood as integral to the daily administration of writing programs and is made even more complex when program assessment is understood as a form of research. Even our leaders seem to be at a loss. For Rose and Weiser (1999), writing program research is taken to be "theoretically-informed, systematic, principled inquiry for the purpose of developing, sustaining, and leading a sound yet dynamic writing program" (ix). For Douglas Hesse (2012), that definition is an Aristotelian act of structuralism. Writing program research, he believes, most benefits by attention to the acts, purposes, and audiences of the programs themselves. Like Polonius echoing Hamlet's sarcasm, different observers see different things in shadowy shapes.

What we might call the definition problem is all the more significant because writing has been so simply defined. In 1963, Albert K. Kitzhaber measured writing performance by elaborate error count, thereby defining good writing as correct writing, a construct few would defend today Kitzhaber (1963). The Braddock, Lloyd-Jones, and Schoer (1963) study defined writing research in terms of limited empirical methodologies, thereby ruling out characteristics of writing not readily measurable. When some writing programs developed as separate entities from the English departments within which they customarily resided (marked by the founding of the Council of Writing Program Administrators [CWPA] in 1976), researchers began anew to define what such programs were seeking to attain beyond measures emphasizing knowledge of conventions. A team of researchers led by Michael Scriven (1981) produced twenty separate technical reports in a Carnegie Foundation-funded attempt to document the outcomes of the (then) Bay Area Writing Project from 1976 to 1979; the findings were inconclusive, though the revised volume provided what the authors called a "handbook" for those doing further research on the topic. The Fund for the Improvement of Post-Secondary Education funded a study culminating in an important volume in 1983 by Stephen Witte and Lester Faigley that focused primarily on a more expansive vision of what a college writing program could accomplish and found the usual empirical study inadequate because the assessment objectives were unclear: "We have suggested that the complexity of writing programs has done much to limit the development of adequate evaluative procedures and methods. Yet the sheer complexity of the thing evaluated is not the only reason the art of writing program evaluation remains in its infancy" (Witte and Faigley 1983, 66).

At the same time, a project led by Edward White and funded by the National Institute of Education employed empirical methods in a five-year effort to determine the writing program features most correlated with effective student writing. Published in a final report (White and Polin 1986), the results presented findings of interest but contained predominately descriptive information about the kinds of writing instruction then most in use. Although research — most notably the meta-analysis by Hillocks (1986) and the taxonomy by Stephen North (1987) — also appeared in the mid-1980s, the frustrating variety of definitions of good writing continued. And, as expected, equally absent were reports of convincing research on writing programs. Recent studies have expanded knowledge of program assessment by attempting to articulate the objectives of writing across the curriculum (Bazerman et al. 2005; Yancey and Huot 1997) and writing in the disciplines (Neff and Whithaus 2008; Poe, Lerner, and Craig 2010). Such specificity has yielded enormous benefits for the design of programmatic assessment, with recent scholarship focusing on the assessment of writing centers through attention to key elements of locally based assessment, including alignment of the program with institutional mission, establishing meaningful outcomes, and communicating assessment results (Schendel and Macauley 2012).

Yet we are not alone in our struggle. Interwoven with dissatisfaction about the inability of our field to come to terms with the success of its own instructional programs is a century's worth of similar expressions of doubt in the nation's educational system itself — especially its ability to produce reform through educational research (Lagemann 2000). If we seek a hallmark document, we...