Education and Goodhart’s Law

I don’t remember when I first heard about Goodhart’s Law, but it was probably several years ago.  But as I think about the higher education in general, and admissions specifically, it seems to be more and more relevant to our daily lives.

Goodhart’s Law was first articulated by Charles Goodhart in a 1975 paper, as “As soon as the government attempts to regulate any particular set of financial assets, these become unreliable as indicators of economic trends.”  This might have been the end of it, but Dame Ann Marilyn Strathurn (as women are wont to do) put it in clearer and more concise terms by saying,  “When a measure becomes a target, it ceases to be a good measure.”

There is a good example of the practical application of this outside higher education (ambulance response time) here.  Or, as Phil Ebersole says elegantly, “The aim is evidence-based policy.  The result is policy-based evidence.”

It used to be, of course, that admission rate was just a number; the total number of admits divided by the total number of applications.  As soon as it took on the mantel of proxy for educational quality, it became a target: Institutions believed they would be better if their admission rate was lower, so guess what?  They started to admit a lower percentage of applicants, but in a surreptitious way: Pumping fake applications into the denominator to change that measure of selectivity.

Other institutions super score ACT or SAT scores, not because it changes the quality of the class, but because people look at average test scores as important.  We could raise the average ACT of our freshman class by 1.25 points if we reported super-scores, but we don’t.  Alternatively, some colleges require submission of every set of scores the student has in her record, but only report the best ones.  Many have raising the mean test score in the freshman class as an objective.  And we see what happens when people have these goals explicitly stated: They meet them, come hell or high water.  Several high-profile institutions have admitted to “reporting irregularities.”

This happens at the trustee level, as well, where people use the discipline of accounting, a reporting function, to manage expectations going forward.  Thus, a discount rate, which is an appropriately acceptable way to measure  past performance, now becomes the target going forward, even though it’s foolish to try to keep the discount level while increasing tuition much faster than inflation and trying to increase or stabilize enrollment. (Note: This is far more complex than the slideshow suggests, but the math simply shows the folly of the attempt.)

And of course, it happens at the high school, where NCLB mandates tests to measure learning, but the learning becomes subservient to mastering the test.  The problem with us chasing our tail, of course, is that we really know no other way: How do we measure something so nebulous and so personal as learning in a standardized way?  And how do we assure taxpayers and other stakeholders that they’re getting what they pay for?

You tell me.