The Problem with Education Technology

TEACHERS VS. TECHNOLOGY — ROUND ONE, FIGHT!

In today's society, college is ambiguous. We need it to live, but we also need it to love. ... Teaching assistants are paid an excessive amount of money. The average teaching assistant makes six times as much money as college presidents. In addvition, they often receive a plethora of extra benefits such as private jets, vacations in the south seas, a staring roles in motion pictures. Moreover, in the Dickens novel Great Expectation, Pip makes his fortune by being a teaching assistant.

It was our last day at the 2012 Conference of College Composition and Communication (CCCC) in St. Louis, the biggest annual gathering of college writing teachers in the country. We were done with our own presentation, and we could eat only so many ribs in a day ... so we were looking through the conference program to see what else was on offer ... when a full-page listing caught our eye. It was for a "Special Session." Something particularly Big and Important. This Special Session, the program told us, would be a "vigorous but civil debate" on Automated Essay Scoring (AES), the technology that lets computers score writing assignments.

We were intrigued. When an academic conference has to specify that an event will be "civil," you know something's up.

The ballroom was vast and long. There were about two hundred seats facing a long table and a giant PowerPoint screen. At one end of the table sat Paul Deane and Chaitanya Ramineini, researchers from Education Technology Services (ETS): the College Board, the minds behind standardized tests like the SAT, GRE, and AP exams, and — along with their for-profit colleagues Pearson and Vantage — the loudest promoters of AES. They were here today to present their newest innovation: the e-Rater Scoring Engine, a computer program that — according to an impressive set of studies — can score a student essay with the reliability of a human.

At the other end of the table sat Les Perelman, the longtime (now emeritus) Director of Undergraduate Writing at MIT — and the best-publicized crusader against AES. He authored the first paragraph of this section, which he fed, along with another page-and-a-half of similar drivel, into e-Rater. And it got a 6, the highest-possible score. The point seemed clear and obvious: machines can't read. They can't understand. And they can't, and certainly shouldn't, replace humans in educating our children.

(Between them, fittingly, sat Carl Whithaus, director of the University Writing Program at UC Davis, who works to integrate AES into human-centered teaching. We'll get back to him later.)

Les was the perfect foil to Deane and Ramineini. They were impeccably dressed; he wore a suit that didn't fit quite right and looked a little sweaty. Their speech was memorized and newscaster-perfect; he spoke off the cuff and made no effort to hide a general-purpose ethnic accent. Their PowerPoints were sleek, branded, and serious; his was homemade and funny, complete with gifs of rabbit holes and the Twilight Zone.

None of this was by accident. They were each playing their parts to a T: the bloodlessly efficient technocrats versus the righteous, rumpled, lone defender of flesh-and-blood teaching. "Vigorous but civil" be damned — this was a brawl. An agon. A pundit war between good and evil, man and machine. We, the human educators of CCCC, sat and winced as the forces of evil took their best shot — and then rejoiced as our champion tore them limb from limb, down to their last unfounded assumption and logical fallacy.

He was, after all, fighting for our lives. Or at least our livelihoods. Humanistic objections to robo-teaching aside, we were precisely those humans AES stands poised to eradicate. AES, which can grade thousands of student essays in mere seconds, could convince a budget-wary public that small class sizes and individualized instruction are unnecessary luxuries. In a political moment where education funding is under constant attack, it's not hard to imagine administrators and elected officials (even the well-meaning ones) using AES as a rationale to lay us off. So when we see Les up there proving that AES doesn't work, won't work, can't work, of course we cheer loudly.

The problem is, outside the cozy confines of CCCC, he's losing. We're losing. Education technology — computer-based classroom instruction, online K–12 schools, MOOCs (Massive Open Online Courses), as well as AES — is all the rage. It's what's getting all the media attention, foundation funding, and government grants. It's the cutting edge, the thing all forward-thinking principals, superintendents, policymakers, and executives (both public and private) are talking about. Les's dogged defense of old-fashioned teaching might make us cheer, but it's making everyone else yawn.

And maybe most frustrating of all, the science all seems to be on their side. Sure, Les's argument makes intuitive sense — how could you trust a program that gave that pile of s#!t he wrote the highest-possible score? — but then again, how can you argue with scores of reliable, data-driven studies?

This is the problem we'll tackle in this book. We'll explain the technology, how it works, and what it does (and doesn't do). We'll explain where it comes from, and how it's come to take the form that it has, in the classroom and in our culture. We'll also explain why the debate over the technology has taken the shape it has — a shape that stops us from understanding the real problem or doing anything about it. Finally, we'll explain what that real problem is, and how we — teachers, students, citizens — can work to solve it.

Because there is a problem. It's just quite a bit subtler, tougher, and more complicated than the standard "civil debate" would have us believe.

THE FIGHT OVER COMPUTER GRADING (STARTED BEFORE COMPUTER GRADING)

Human versus machine. Good versus evil. Teachers versus technology. Kids versus computers. This is a Big Issue, and not just for writing teachers.

A few weeks after the conference, it made the New York Times. "Facing a Robo-Grader? Just Keep Obfuscating Mellifluously," read the headline. The story, by longtime Times reporter Michael Winerip, starts with a recent study that "concluded that computers are capable of scoring essays on standardized tests as well as human beings do" — at the rate of 48,000 essays per minute.

It gives some space to ETS representatives, including Paul Deane. But most of the story belongs to Les, who demolishes...