Education is an Industry

"education is an industry" says Stephen Moore of the Wall Street Journal. That's not the title of his op-ed, or even the point of his op-ed. In fact, that's just an unimportant introductory phrase of a sentence in a paragraph casually criticizing education in the U.S.

And that's why it's the most interesting part of his whole piece.

He doesn't argue that education is an industry, but treats it as a completely unquestioned assumption. Our ability to solve problems is severely limited by how we frame those problems, and his piece shows that education is now completely framed as a problem of industrial production, a problem of business, and economics.

And nothing really to do with human beings.

To put it in context for those too lazy (or caught by a paywall) to read his piece, here's the section that caught my eye:

Where are the productivity gains in government? Consider a core function of state and local governments: schools. Over the period 1970-2005, school spending per pupil, adjusted for inflation, doubled, while standardized achievement test scores were flat. Over roughly that same time period, public-school employment doubled per student, according to a study by researchers at the University of Washington. That is what economists call negative productivity.
But education is an industry where we measure performance backwards: We gauge school performance not by outputs, but by inputs. If quality falls, we say we didn't pay teachers enough or we need smaller class sizes or newer schools. If education had undergone the same productivity revolution that manufacturing has, we would have half as many educators, smaller school budgets, and higher graduation rates and test scores.

He's not saying anything terribly original here (education, after all, is just a token example of his overall rant that government is unproductive), just expressing what is now the winning framing of the problem of public education. This framing wins in the popular mind because it promises simple answers to complex problems and appeals to common sense. To grasp what's wrong with it, you have to take a step back and look at how the problem is being framed, which is a hard mental step for many.

To break out of framing, it's best to first locate an absurdity. For example, it sounds real reasonable when he says that if education had undergone a "productivity revolution" we would have half as many educators. OK, so instead of "half as many educators", why didn't he conclude that all our students would be completing high school at the age of 12? Or that our students would all be speaking twice as many languages when they graduated high school? Why does "half as many educators" sound reasonable, while learning twice as fast does not? It's because my absurd examples remind you that we're talking about real human kids, not industrial products.

It's hard to see how there could be any improvement in U.S. education while the problem is framed as it is. Note the framing implicit in the phrase "standardized achievement test scores". I could replace this phrase with "arbitrary test scores" and factually be talking about the same tests as him. You see, these standardized tests are indeed standardized, they're just not particularly rational tests. What do these tests measure? Well, to find out what a test measures, you have to have some (hopefully more objective) measure to compare it with. For example, you could compare the results of these tests with how much money the test taker earned later in life, how likely they were to never be in jail, how much taxes they paid, etc. If you could agree on what the goal of education is (which Moore implicitly assumes to be simply manufacturing a product -- a student with high test score), you could try to measure outcomes later in life, compare them to the test scores, and then start to learn whether the test had any relevance to those goals whatsoever. But you can't even have that conversation, since the problem is framed to assume that everyone agrees that "standardized achievement test scores" are objective and desirable measures of education.

Pay for Performance

This standard framing of the education problem immediately leads to the commonsense concept of "pay for performance." The framing implicitly excludes the idea that there is any doubt about how to measure the performance we are paying for! After all, we have tests that are "standardized" and apparently (so the framing says) have something to do with "achievement". Once again, the easiest way to break the framing is to note the absurdities. If "pay for performance" is such a good thing, why is it not applied to the companies who get paid tax dollars to create the tests? For example, during the years that the SAT was highly incorrect at predicting which black students would be successful in college science or math (the future winners tended to get low SAT scores), did they offer refunds to black students for that failure? Of course not. The framing says that the current tests are the goal, so the difficult question of what the goal of education should be cannot be discussed.

And of course, in the extreme, "pay for performance" leads to "getting rid of bad teachers". Again, to break the frame, you have to notice an obvious inconsistency. Such as, who exactly was it that hired these "bad teachers"? It is certainly much more productive to find out who is so incompetent at hiring teachers and get rid of them than it is to hire teachers, have them fail (with our kids!) for a year or two before firing them and starting another (expensive) hiring cycle again. But once again, the framing insures such commonsense thoughts will never arise. Education is an industry, students are products, teachers are assembly-line workers whose pay needs to be driven down to increase efficiency. OK, technically he's arguing they're assembly line workers who should be able to assemble twice as many units (students) per year, but it's the same thing. Once you frame it as a problem in economics, then cutting teacher's salaries in half is quite equivalent to having them be twice as productive.

This framing of the problem has been so successful that I often even see high-level teacher representatives unable to articulate what might be wrong with "pay for performance". Suppose that I am a teacher and you inform me that my pay depends on my students' standardized test scores at the end of the year. Here is what you are actually telling me:

• Scrap Struggling Students Is your kid having a bad year? Tough luck, then, because my time is (mathematically!) better spent raising scores on students who are closer to the mean than on investing a lot of time in your little loser. Better yet, maybe just a wee bit of negative feedback (it need not be verbal, written or even overt!) will get your kid to drop out -- that's really the fastest way your underperforming kid can get me my performance incentive raise! I think Texas invented this neat accounting trick, AKA "No Child Left Behind... on the Books".
• Teach to the Test Well, duh. If my pay is determined by one big, important test, guess what I'm going to teach. I guarantee I can raise any kids math test scores on a standardized test. I also guarantee they will learn to hate math, and be unlikely to ever go into science, engineering, or anything else that has Calculus I as a gate-keeper. If you want them to actually learn the art of math, well, that takes time and I have an assembly line to keep up with.
• Sandbag Suppose you get clever and start grading me on "improvement" rather than just a single test outcome? Easy peasy. Now I just don't teach the kids anything about the baseline test. In fact, if possible, I'll even make sure they don't know there's a test coming. Once the kids flop on that, then I'll start teaching to the test real hard, train them in all the little test-taking tips, push them to get a good night's sleep the night before and, voila! Improvement! Ka-ching.
• Cheat The one thing that is a 100% predictable outcome of judging teachers based on narrow standardized tests (usually created by for-profit companies with zero accountability for their performance), is that you will increase cheating. So, it should come as no surprise if Michelle Rhee's tenure came with cheating on tests. It would be very odd if there were no increase in cheating. If you take professionals, grade them on arbitrary narrow measures that are distant from the complex, professional goals they trained for, you will get rebellion and cheating. It's not because teachers aren't professional -- just the opposite. This was, after all, the plotline of just about every M*A*S*H episode. You can't cheer when Hawkeye flouts military rule to benefit a wounded soldier and then boo when teachers cheat so they have more time to teach your kid something really useful.

Pay for performance sounds like a wonderful thing, just so long as it is being applied to someone else. Why aren't op-ed pieces being produced twice as fast these days, or with twice as much quality? Couldn't we improve the WSJ's editorial page by switching the editors to "pay for performance"? We have the technology (technology being the other baseball bat people like to swing at students' heads) to have each op-ed piece subjected to a variety of tests, including letting readers vote on what's boring, or objectionable. Somehow, I doubt the WSJ will apply any of the prescriptions they offer to the educational system to themselves.

The Lessons of Business

This is, after all, the WSJ I'm poking fun at. Isn't it unfair to mock their grasp of education when their specialty is business? Actually, if the WSJ could learn the lessons of business, they would understand education much better. After all, how could someone at the WSJ not have grasped the lessons of Dr. Deming?

Deming taught U.S. businesses how to improve their product quality, in part by making life better for workers. He went to Japan and was a key player in turning their auto industry into the monster that crushed Detroit. Returning to the U.S., Ford was really the only one of the Big Three to take him seriously. Ford is also now the closest thing we have to a healthy, independent, auto maker. Anyone writing for the WSJ surely knows of Dr. Deming and knows that one of his famous Seven Deadly Diseases for business was "Evaluation by performance, merit rating, or annual review of performance"

Deming also taught (partly via a hands-on "manufacturing" exercise designed to gently humiliate the participating CEOs) that you can't say anything about how good a job a worker is doing if you don't know what the standard variance is in the job. Notice that Moore's comments and the standard framing of the education problem both ignore the possibility that students are not all the same. In fact, when Moore talks about "inputs" to the education process, he doesn't mention students at all! Deming spent a lifetime trying to stop unproductive stupidity such as rewarding one teacher because they happened to get a batch of students well above the median (but within typical variation) and punishing another teacher because they happened to get a batch well below the median.

Or, even if a WSJ editor had no grasp of Deming's ideas, they should surely have acquired a visceral feel of the dangers of incentive pay. CEO stock options, started as a "commonsense" incentive, have turned into the most ludicrous, obscene, and counter-productive financial transaction in the country. Even working at the WSJ, how does one become deadened to spectacles such as Alan Fishman's $20 million dollar payday for spending 17 days overseeing the bankruptcy of Washington Mutual? This sort of thing makes it hard to take seriously a business newspaper complaining about the inefficiency (keep in mind that by "inefficiency", they partly mean "keeping people employed") of government.

After All, Psychology

In my book, I argue that psychology represents the low-hanging fruit for improving programming, simply because it is the low-hanging fruit in every industry. Our brains are wired for "common sense" appropriate to the world of a million years ago, not to modern problems. Every single person is born with a brain wired to think like Moore, to think that things like "pay for performance" just are obviously good sense. It takes (ironically) education to overcome our "common sense" and grasp why incentive pay can produce exactly the opposite of its intended outcome. It takes education to be able to break out of a frame that has been constructed to only allow poor solutions to a problem.

One reason Deming went to Japan was his experience in the U.S. He found that he could go to a company, train the people, and the quality results would follow. But when he went back to the same company later, higher-ups would have replaced his methods with "common sense" approaches like pay for performance. In Japan, he was able to teach people at the top. Eventually, he made a rule of not helping a company unless the most senior executives would take the training first.

Understanding and overcoming the situations where "common sense" is just wrong is the domain of psychology. It will always be the low-hanging fruit for improving any complex human endeavour because our cognitive limitations have to be overcome one person at a time, the slow, human way.