More on Non-Cognitive Variables

I’m back from the USC CERPP Conference on non-cognitive variables in the college admissions process, where I gave a presentation on the results we’ve seen at DePaul, and offered some commentary on the whats and whys of what we as a profession are doing.

In a nutshell, here’s what I said in my presentation:

Most admissions people know that the things we collect at the point of admission do only a so-so job of predicting how well students will do in college.
We also find lots of kids who have “something” that we think will help them despite academic records that might suggest otherwise.
Many of the things we do look at tend to correlate with income. Correlation does not imply causation, of course, but regardless, if we consider them important, some students benefit while others don’t.
The world’s great thinkers have struggled with cause and effect, and especially with prediction inside a complex system. This is especially true when we suspect there are variables yet undiscovered that might lead to insight.
We may have the “poodle problem.” That is, if we have a poodle, we know it’s a dog; if we have a dog, we can’t be sure it’s a poodle. In the same way, if we say that students who succeed have certain traits, that does not mean that all students with those traits will succeed.
Non-cognitive variables, as we currently understand them, make our jobs a little, but not a lot, easier. But a) that doesn’t mean they won’t eventually be more helpful especially if we find better ways to measure them, and b) it certainly doesn’t mean we shouldn’t stop trying.

Some other things that stick in my mind as I get over jet-lag:

We all owe a great deal to Bill Sedlacek, for his amazing pioneering work in this field.
The Morehead-Cain Scholarship Program seems to be way ahead of the rest of us in understanding this stuff.
We saw research that suggested that the LSAT may predict who will be a good law student, but actually is a negative predictor of who will be a good lawyer. Really amazing stuff.
Challenging the ridiculous premise that “GPA + test scores = merit” is perhaps our greatest challenge as a profession. We have a long way to go in effecting the collective belief.
As is almost always the case, the people I meet at conferences like this are interesting, smart, and really dedicated to their profession and in making the world a better place.
It was good to hear David Coleman speak about change in the College Board, and I’m glad they’ll be releasing data to colleges to support our work, but I didn’t hear enough to convince me over a talk at dinner.
In my opinion, the premise upon with Richard Sander based his research for “Mismatch” could have and should have been vetted by talking to someone who can help him understand things more clearly. (He’s both an economist and a lawyer, two professions, I’m afraid, that tend to believe they can look at anything and figure it out with just a rigorous intellect.)

On that last point, let me give you an example: Sander posted a chart (a really horrible chart, by the way) that showed widely varying completion rates for students who selected STEM (Science, Technology, Engineering, Math) majors. But he fails to see his own fallacy of equivocation: If we lump all “students who select a STEM major as a freshman” into a group, we already fail to recognize an important concept: That wealthier students, from more affluent schools, with college educated parents, and access to better college guidance, have selected STEM majors after a fairly rigorous sorting process. The advantaged ones who probably should not be in STEM fields have weeded themselves out; for poorer kids with fewer advantages, the freshman courses serve the purpose good counseling could have done. But I don’t see any control for that fact, a fact that would have been obvious to anyone who’s done this job for a while. (Note: I’ve not read his book, and it’s possible that somewhere in a footnote this is addressed, but he made no reference to it in the presentation.)

Overall, this was one of the best conferences I’ve been to. It’s great to talk about and hear about something other than the same issues we hash and re-hash on an annual basis. I hope next year’s is equally good.