A Decision-Making Process called PUMI

Key Vocabulary

• Assessment or testing

• Assessment lens of promise

• Assessment lens of deficit

• Translanguaging

• PUMI (purpose, use, method and instrument)

PUMI Connection: Purpose, Use, Method, Instrument

This chapter introduces the concept of PUMI – 'purpose, use, method, instrument' – which will be used in each chapter for the remainder of the book. PUMI is a decision-making process to help stakeholders make better decisions about assessment for emergent bilingual (EB) students. The author recommends that when you are unsure about what questions to ask and what is important, start asking PUMI questions. What is the purpose? How will the results be used? What is the best method? What is the best instrument? Finding the answers and understanding PUMI help slow down the process of assessment and lead us to using fewer and better assessments.

This foundational textbook has three primary objectives. First, it helps teachers and administrators understand the challenges with assessment and accountability for EB students that dominate the field today. Second, it prepares teachers, administrators and leadership teams to make decisions about how to use and select appropriate assessments for EBs. Third, this book prepares educators to advocate on behalf of EBs in regard to appropriate test-use policies and practices.

It's a Continuum: Promise to Deficit

Valid assessment for EBs is a complex scientific challenge, especially in a monolingual schooling context. The various approaches to assessing EBs in the field today draw on contrasting views of assessment and bilingualism. This section briefly reviews a continuum of views, followed by guiding principles that teachers and administrators can draw on as they make practical decisions about assessing EBs.

There are many ways to view the assessment of EBs, and these can be best understood as a continuum – a wide range of approaches with extremes on either end – ranging from deficit to promise. Promising approaches to assessment highlight what the student knows and can do relative to multiple measures; on the other hand, deficit approaches highlight what the student doesn't know, usually relative to one measure. What is especially challenging is whether and how educators can look at EBs through a lens of promise within an accountability system focused on what children cannot do (Figure 1.1 and Table 1.1). Understanding this continuum will prepare the ground for better EB assessment policy, practice and advocacy.

The lens of promise is grounded in the ideas of dynamic bilingualism (García, 2009) and sociocultural assessment (Stefanakis, 1999), and is typically used in assessment courses and popular textbooks to guide educators in how to assess (and instruct) EBs within a meaningful and culturally relevant context. However, more often than not, and in contrast to a promising lens, deficit views of assessment dominate the policies and accountability system under which educators must perform. Deficit and promise are strikingly different, and educators are required to negotiate them in a public-school setting in order to maintain good evaluations of their own teaching and do what is best for students. These two views are presented here as a continuum; that is, one end of the continuum differs extremely from the other end, but points near one another on the continuum may not be that noticeably different. These two views are not a dichotomy – mutually exclusive of one another, where educators must choose one view or the other. What is most common in schools today is a mix of both deficit and promising assessment approaches; state and district accountability systems are more often connected to looking for deficits or areas where children are lacking, whereas classroom assessment is more connected to the approach of promise. Usually, teachers negotiate both views. Table 1.1 illustrates in more detail both ends of the continuum.

Assessment vs. Testing

Oftentimes in conversations that take place in schools, the words assessment and testing are mistakenly used synonymously. Assessment is a much broader concept than testing and can be thought of generally as the use of information from various sources to make decisions about a student's future instruction/schooling. Testing, on the other hand, is a measuring instrument that produces information we use in assessment. A test can be thought of generally to be like a measuring instrument such as a measuring cup or a scale or a tape measure ... used to measure ideas to help make decisions in schools. The example given above is a very simplified view, just to point out the difference between assessment and testing. Measuring flour in a measuring cup is much easier than measuring language in a child emerging as bilingual.

Support for EBs in Measurement Community

It is important to note that stand-alone tests themselves are not bad. However, with the increase of testing and the high-stakes decisions made from test scores, most educators have become very frustrated with the way that tests scores are used. The frustration is usually with the test-use, not the test, and especially for groups of students like EBs. Many times, policymakers, legislators, school boards, etc., use test scores for EBs in wrong – or invalid – ways. Examples of this from current practice include using achievement test scores to judge language ability (wrong construct), using one test score only to reclassify an EB (never use one score to make a big decision) or using test scores from a test that EBs cannot read very well (not a fair measure). It is important to point out that the educational measurement community does not support these types of test-use practices, and they do in fact support promising practices for EBs.

The term 'measurement community' refers to measurement scientists who have studied educational measurement, topics such as fairness and validity, for several decades. The three organizations that dominate the science and use of test scores in the US are the American Educational Research Association (AERA), the American Psychological Association (APA) and the National Council on Measurement in Education (NCME). The study of (educational) measurement is basically the practice of assigning numbers to traits like achievement, language, interest, aptitude and intelligence. Oftentimes, measurement scientists design studies to try to validate whether those numbers really represent the trait and as a result, they make recommendations about how to use test scores; however, it is up to policymakers to write good policies about how to use test scores. This is usually where 'a disconnect' occurs, leading to deficit assessment practices.

A good example of the increased attention toward EBs from the measurement community is an important publication...