Do Notre Dame Football Graduation Rates Prove the Value of Non-cognitive Variables in College Admission?

A recent article in USA Today lauded the ways in which Notre Dame football is Number 1 in graduation rates of its players. And of course, they’re now Number 1 in the AP Poll and BCS rankings for College Football, too, a rare accomplishment that seems to make the always-proud alumni base even more sanctimonious than usual. (Note: This link will be obsolete as games are played.  Here’s a screenshot of it as of November 28, 2012.)

But this is not about those people who allow cult-like pride and slavish devotion to a non-existent ideal to interfere with reality. And it’s not really about Notre Dame, either. It’s more about selective college admissions, graduation rates, and what “admissions standards” really mean.

It’s widely acknowledgedeven by people within an institution–that the academic profile of athletes as we traditionally measure such things is lower than that of the non-athletes in the freshman class. It’s true everywhere.  No news here.

But does it seem at all odd to you? If a highly selective institution (and I include the likes of Northwestern, Stanford, Duke, and other places that combine big-time athletics with high levels of selectivity and the accompanying graduation rates) publicly opine (either overtly in their words or covertly in their actions) that only the best students as measured by SAT and GPA can succeed, how is it possible that so many students who are at least one–or maybe two–standard deviations below the mean manage to do so well? And, you might ask, how can they manage to do so well while committing to what must be the equivalent of a 40-hour work week?  Conversely, if many of the low-income students who don’t measure up on traditional measures promised to spend an extra 40 hours per week studying, could they graduate too? (Many of the most selective places in the country don’t admit poor students, largely, I believe because poor students usually score lower on the SAT or ACT.)

It’s true, of course, that we don’t see final GPA’s of the athletic students (I’ve never liked the term “student athletes”), so maybe this is where the disparity comes into play. But assuming that graduation is the real threshold, one of several possibilities might occur to more cynical readers:

  • Support services for athletes are extraordinary
  • Someone else is doing the work
  • Athletes take easy classes sanctioned by the university
  • The university is really not as rigorous as it claims, and anyone could graduate

But being the cheerful optimist that I am, something else has occurred to me:

What if the thing that really gets you through college and through life is not just intelligence in the way we traditionally measure it? What if it has to do with things like leadership, drive, determination, motivation, goal setting, moral support, and dozens of other non-cognitive things we can’t even describe?  What if that intangible “it’ that admissions officers see that makes them take a risk on a candidate means a lot more than we give it credit for?  It’s almost like Bill Sedlacek was right.

That’s what I’ve come to believe. Academic intelligence and cognitive ability are important, of course. But if you believe Al Maguire’s “The world is run by C-students,” or Woody Allen’s “Eighty percent of life is just showing up,” you begin to wonder whether any long-held belief about the way we do college admissions is meaningful.

I’ve been called an instigator. It’s also been said I love to stir the pot. Tell me what you think.

What if You Threw a Scandal, and No One Cared?

I’m not even going to link to the stories, as they’re so abundant and common: Over the past several years, many colleges have been caught (or sometimes admitted without getting caught) doing things to inflate the profile of the incoming class.  Usually, this is the freshman class and SAT manipulation (Emory, Baylor, Claremont McKenna); sometimes it’s class rank or academic accomplishments of freshmen (George Washington University and Iona); and occasionally, it’s been law schools and their first-year class (Villanova and The University of Illinois.)  These are pretty clearly unethical, and almost always frowned upon, except if you’re  the former president of GWU Stephen Joel Trachtenburg and you don’t understand what all the fuss is about.  Because, apparently, reporting actual historical data is just like forecasting whether it will rain on Alanis Morissette’s wedding day. Neither of which, by the way, is ironic.

This doesn’t even begin to take into account things colleges do to inflate scores that are generally accepted: Superscoring ACT’s in order to create composite scores that don’t exist; requiring all instances of the SAT a student has taken, but reporting only the highest of each subsection; deferring wealthy but weaker students to spring admission, where no one ever looks at the freshman profile, or giving preferences in admission to applicants from schools that don’t rank because, hey, if a student doesn’t even have a rank, how can I report it?  And, does anyone know what an applicant really is these days?  Some places count as an applicant anyone who clicks on a special “Priority” application link in an email; most of those who don’t get admitted (thus lowering the admission rate) are in this group.

Even though we don’t do any of those things, I’m not being judgemental.  I happen to work for a place that takes internal measures of academic quality very seriously, but doesn’t worry too much about inputs.  In other words, I’m very lucky.  But if my president or trustees told me to do those things, and if I wanted to keep paying my mortgage, I’d have to think long and hard.

US News and World Report, the oft-cited villain in the admissions arms race, just took some unprecedented action, removing GWU from its online rankings. But what if, come a year from now, none of this mattered?  What is students and parents still considered GWU and Emory and CMC and all those places for what they are, not for what a magazine says they are?  What if people cared more about what happened in the classroom than what happened in the four years before a student was admitted? What if we went back to 1977 (the year I graduated from high school) to a time when no one really knew or cared about all those numbers? Or, when no one cared about what you wore for your senior portrait, either.

The proverbial Genie is out of the bottle, of course, so none of this will happen.  But would the whole admissions profession be better if it did?  Would students behave differently?  Would parents behave at all?

What do you think?

Idea Block, Twitter, Traffic Lights, and Other Signals

I had a boss who once talked about “sending ideas out to the universe.”  It’s poppycock, of course, if for no other reason than the universe doesn’t care.  But that doesn’t mean it won’t work sometimes, I suppose. Yesterday, I tweeted this:

And this morning, the Twitter-verse unknowingly responded with this:

Which gave me–as only Twitter can–the ability to legally eavesdrop on three colleagues: John Lawlor, Deb Maue, and Chris Lydon.  I know exactly what the conversation means: Measuring the quality of the educational products of a college by measuring the freshman class is like measuring how good a basketball team is by evaluating the average height of its players.  It might work, on occasion, but that doesn’t mean it’s right.  However, I responded to all three, suggesting that in fact graduation rates were inputs, in a weird sort of way.

That probably would have been the end of it, but it got connected to something I frequently see on my walk between Union Station and my office, about a mile each way.  And it is especially common as it gets colder and people are eager to get indoors.

Here’s what happens: You’re standing at a corner with twenty other people waiting for the light to change, and someone notices no traffic coming, so they start to cross against the light.  Others follow, even though the sign is clearly “Do Not Walk.”  People come from the other direction, and, seeing the throngs of people crossing, assume the light says “Walk” and proceed to do so.  But by this time, a car or truck or two has come barreling down the street, and I almost witness a pedestrian hit by a vehicle.  What happened?  It’s clear that the pedestrians have equated large groups of people crossing the street with a “Walk” sign.  And usually, that signal is right; occasionally it’s not, and trouble ensues.

Thus, a blog post. And the end of Idea Block. And the head knocking resumes.

If we attempt to measure the value of an institution by its outputs (graduation rates) we are really just confusing signals, like pedestrians on Jackson Boulevard in Chicago on a cold morning. We’ve become so used to the mix up of inputs and outputs that we forget to look at the real signals; because in a way, inputs and outputs are the same things.  Don’t believe me?

Take a look at this screen shot of IPEDS Data visualized.  It shows test scores on the x-axis, and graduation rates on the y-axis. Note how they line up?

That alone would be enough to make the point, I think.  But play with this visualization by clicking here.  See if you can find any type of institution where it doesn’t hold: Urban, rural; any region; any religious affiliation;  public or private.  And notice two more things:  The color of the dot (which represents a single institution) shows the percentage of freshmen with a Pell Grant.  And the size of the dot shows the rejection rate: More selective institutions have bigger dots.  Then, just for fun, filter to institutions that don’t’ rely on student tuition to manage the budget.  Just pull the top slider down to 50%.  See who’s left.

Then ask yourself: Is a high graduation rate a function of what goes on inside the institution? Or could it be a function of selectivity, test scores and family income (which are pretty much the same thing) and resources?

Would love to hear what you think.

Life, and EM: A Series of Trade-offs.

There are very few people who understand that Enrollment Management is, at some level, an exercise in managing trade-offs.  Even though the old Michelob Light commercial suggests that you can have it all, in fact, you can’t.  And in reality, if you could, you wouldn’t be working in higher education. (Those of you who know me also know I like good German and American beer, but I’ll keep my comments about Michelob Light to myself for now.)

So, helping people understand trade-offs is a critical component of working in Enrollment Management:  If you want to push up or down on quality, quantity, diversity, or net revenue, the market is going to be more than happy to push back on you, often harder and more dramatically.  But very few people outside of Enrollment Management understand this; those who do grasp the concept of trade-offs probably don’t have the time or the inclination to dive into the details to see the nuances.

My job is essentially trying to hit a sweet spot: Managing to generate enough net revenue to pay the professors, heat the buildings, buy computers, and keep the library stocked with academic journals and important books; keeping quality as high as we can in light of the need to pay the bills; and not ever giving up on a critical component of our mission to educate those whose economic situation might not normally assume a private university education, because offering low-income students anything less than a top-quality education only adds insult to injury. Keeping these things in balance is vital to accomplishing what we set out to do.  And we re-invent the way we do it every year, because the number of students in the world is fixed, and competition is pretty fierce.  On top of it all, every university has a different recipe for success.

Historically, we’ve managed this delicate and ever-shifting balance by using SPSS and Excel to examine the relationships between and among the variables we are interested in; typically, we spend several days a year doing nothing else, and it often involves Powerpoint decks of literally hundreds of slides.  When your attention span is as short as mine, I guarantee you lose something important while day dreaming.

So, for internal use and to illuminate the balancing act, this year I took four years of data and rolled it into my favorite visualization tool, Tableau Software. In the interest of full disclosure, I’ve served on their Customer Advocacy Board, because I’m a fan, not because of the free T-shirts, or the beer I’ve been promised by my former account manager for five years now!  I have no financial interest in the company.

The data is confidential, of course, so I can only show you a screen shot, which has been sanitized by removing the values and the axis labels.  But look at this: With just a click or two, you can make a shallow or a deep dive to see the give-and-take between and among a handful of variables: Which students have the highest GPA?  How much do test scores vary by financial need?  Are men or women better students (as if we don’t already know the answer to that one!)  What percentage of our first-generation college students are from Illinois? Who helps us accomplish our mission? Who helps us pay the bills?  Which college has the most attractive students? OK, perhaps that last one is not in the data set.  But you get the idea.

If you work in EM, you owe it to yourself to explain to your campus community the ins and outs of your profession; if you work in higher ed but not in EM, you owe it to yourself to educate yourself about how these important variables relate to each other.  How you do that is up to you, but I strongly recommend against a 247-slide presentation.  You can do better.

Test-optional admissions: A year hence

If you’re a visitor to this site, you know about DePaul University’s decision to become test-optional for freshman admission.  With just a few days to go before our final, official count of freshmen (the fall census), the results from our first admissions cycle are interesting:

  • About ten percent of all applicants applied as test-optional
  • About three percent of all admits applied as test optional
  • Just under five percent of all enrolling students applied as test optional

As one faculty member put it, “So this wasn’t such a big deal, then.”  To which I politely replied, “Well, I hate to say I told you so…” (and of course, I love to say “I told you so.”)

Taking both a higher view of, and making a deeper dive into the data suggests some responses to our early critics:

  • This was a record year for applications to the freshman class
  • It will be the largest freshman class in history; we had predicted our yield to go down, but it actually stayed exactly even with last year (22.9%)
  • The average GPA of enrolling students is at its highest point ever
  • Even once we add in the test scores of the students who applied test-optional (we are collecting them for research purposes) this year’s ACT average will be at a record level (or on par with last year, if not quite a record)

In short, those people who believe a test-optional approach stains our academic reputation seem to have little evidence to back that stance up.

When you start sorting out the test-optional applicants, however, some more interesting patterns emerge:

  • Just under half were White
  • At every High School SES (Socioeconomic Status) band, the test-optional students had higher GPA’s than their counterparts who submitted test scores.  For the students from the lowest SES Schools, the average GPA was a hair under 4.0 (adjusted to fit on a 4.0 scale)
  • Their ACT Scores were as high as 29
  • 35% came from the top 1/8 of the High School Graduating Class; among low SES Schools, this number was 71%
  • Almost 60% came from the highest third of SES High Schools
  • They were distributed mostly toward the middle of our overall academic distribution: Not the superstars, but clearly not the last ones admitted to the class, either.

Now some important research begins: We’ll carefully track the progress of these students as a group, but also individually to see if they progress toward graduation in ways that other students like them (test scores, GPA, strength of curriculum, family income, first-generation status, or ethnicity) do.  Based on results at other universities who have done this, we’re confident those results will be good; if not, we’ll figure out where the gaps are and respond appropriately.

All told, this has been a liberating year after almost 30 years of doing admissions and enrollment work; doing this research, connecting with other people who feel the same, and just thinking about students in more dimensions has been quite a boost.

We’re not out to change the world, but to maybe make opportunity for kids who otherwise might not have thought it possible.  Maybe one of them will end up changing the world.  As with everything, time will tell.

Bier, but not in a Higher Education Context

Occasionally, I write about other things: Things outside the world of Higher Education. This is one of those times.

Today, I’m writing about my favorite Bier style and a Dan Fogelberg song, and in the best tradition of John Donne and the metaphysical poets, I’ll try a little discordia concors as a diversion.

I love a good Bier, even though I only drink about two a week. My favorite style is Märzen Bier, a German style known for its copper color, tan head, very smooth caramel malt flavor, and its low bitterness.  For me, that last part is especially important: Although you can’t really brew Bier without hops, a tendency in the last decade is to see just how much hop bitterness you can cram into a Bier (another is to try to make Belgian styles.  That’s a rant for another day).  Some Bier geeks like to find the hoppiest Bier they can, and lots of microbreweries are catering to them.  I just don’t like Bier with high hop profiles, but I do like this line from a New Yorker article about extreme Bier as it sums up my opinion perfectly:

Garrett Oliver, the brewmaster at Brooklyn Brewery, told me recently. “When a brewer says, ‘This has more hops in it than anything you’ve had in your life—are you man enough to drink it?,’ it’s sort of like a chef saying, ‘This stew has more salt in it than anything you’ve ever had—are you man enough to eat it?’

So, this is my favorite time of the year, in part because I love cooler weather, but mostly because Märzen Bier is now available in stores.  You probably don’t know what  Märzen Bier is, unless I call it by its more traditional name: Oktoberfest beer.  There are lots of good ones out there: Paulaner, Hacker-Pschorr, Great Lakes, Samuel Adams, and Köstritzer, to name a few, and others, like Spaten, Ayinger, and Erdinger that other people seem to like more than I do.

In the 1800’s in Germany, when Märzen became the de facto Oktoberfest style, it was seasonal because there was no refrigeration.  You can read about that here. I’m glad to say we’ve come a long way in 200 years, and you can now brew and lager Märzen all year long.  But the Oktoberfest name makes it sound like you’d only want to drink it in the fall.

It occured to me that it suffers the same fate as the Dan Fogelberg song, Same Old Lang Syne. You hear it every year at Christmas just because the song’s story line takes place on Christmas Eve. But the song is not about Christmas.  Similarly, we get Märzen in the Fall because it’s been labeled Oktoberfest Bier. But it’s delicious all year long, and I find it extremely refreshing even on a hot summer evening.  If you don’t believe me, ask Gordon Biersch, the restaurant where a waitress told me that Märzen is the biggest seller all year round.

Now, I don’t really care if I hear the Dan Fogelberg song in July.  But I’d like to get Märzen available more widely all year long.  Just stop calling it Oktoberfest Bier, and see what happens.

Who’s with me?

What Laymen and the SCOTUS Need to Know About Admissions

Note:  Although I’ve said it elsewhere on this blog, I want to emphasize that these opinions are mine.

Ever since the Michigan cases, and now with the Fisher case before the Supreme Court, something has bothered me: The way the average person just does not understand how college admission is done.  I suppose this is true of every profession: I have no idea what an accounting job or a chef’s job is like, for instance.

But just as we don’t choose Supreme Court Justices by looking at GPA and LSAT Scores and Bar Exams, neither do we make an admissions decision based on two simple criteria.  Every case in which someone says another “less qualified person” was admitted to his or her detriment misses this point completely.  And as my old logic professor used to tell us: Always attack the premise.

So, as we were thinking about joining an Amicus Brief with several other Catholic Colleges and Universities, I tried to put into writing what I’ve learned about admissions in the last 30 years.  Here goes:

  • At its core, an admissions decision suggests that someone or some group of people at the university has determined that the student is capable of doing the necessary college-level work to graduate with a bachelor’s degree.  Without this, any other discussion is moot: Admissions professionals consider it abhorrent to admit a student they deem incapable of succeeding.
  • At the same time, statistical models are really not as good as people might believe in predicting the extent of college success.  While the overwhelming majority of students admitted mange to stay above the threshold of satisfactory academic performance, there is of course wide variation between those who perform at a C (2.0 level) and those who perform at a A (4.0) level.  (And given central limit theorem, it will never be possible to enroll a class of students who all perform “above average.”) Using any combination of academic variables we collect at the point of admission, we can only get a statistical r-squared of about .2, meaning that only 20% of this variance is explained by the inputs we presume to be telling; the rest is clearly determined by other factors not deemed academic in nature.  This is true almost everywhere, not just at DePaul.
  • Thus, the definition of “Who is qualified for admission?” seems to be essentially unanswerable by the two simple things many deem to be the sole criteria: HS GPA and test scores.
  • This is especially telling when a college or university has to deny students who, based on simplistic criteria, appear to be more academically qualified than those who are admitted. When you only have so much room, you’re faced with admitting a whole group of kids who look and act alike, or mixing it up a little to improve the educational environment.
  • And additionally, such choices are compounded when an admissions officer attempts to measure potential rather than accomplishment (which of course presumes that “accomplishment” can be standardized across 35,000 high schools and hundreds of thousands of teachers offering millions of sections of classes.)  Obvious inequality of opportunity makes measuring either prior academic achievement or potential to succeed virtually impossible.
  • The history of college admission, however, is not one of making simplistic decisions based on two variables (N.B. that some state schools are mandated to do exactly this, however).  Admissions officers have long sought additional information (writing samples; letters of recommendation outlining, among other things a student’s learning styles and ability to contribute to the classroom and the community; out-of-class accomplishments) to round out the picture of the students they admit to the university, because a university is not solely an enterprise dedicated to passing information to those who are ostensibly most capable of receiving it.  John Cardinal Newman, for instance, in “The Idea of a University” and later Jaroslav Pelikan in “The Idea of the University: A Reexamination” both suggest that a university is a community that develops a wide variety of people for a wide range of outcomes.  Newman even suggests that only 1/3 of university education occurs in the classroom, suggesting that the remaining two-thirds come from self-directed study and other students.  Other factors, often far removed from academic realms (athletic ability, legacy status) are often considered in light of their effect on the total university community.
  • Beyond the core decision, however, one must consider the particular university’s mission or its raison d’etre.  Typically, the mission does not just stipulate why a particular university exists, but also articulates certain obligations, for instance:
    •  To the students it serves.  At DePaul, this ranges from traditional liberal arts programs to professional programs with career focus.  The university has an obligation to educate students who come seeking broader intellectual exposure or top-quality professional preparation, or both; all of which require exposure to a wide range of ideas and people.
    • To the community it serves. The mission statement of DePaul articulates a symbiotic relationship with Chicago, a diverse, global city, in which the university draws from, and contributes to, the city in an attempt to improve both.
    • To the market for its graduates. The critical underpinnings of financial and academic success of DePaul are based on the university’s ability to produce graduates who are ready to work in a demanding, diverse, changing society and workplace.
  • Thus, admissions officers take into account a wide variety of factors when considering an individual candidate, because admissions strives to create a campus community best aligned with the university mission.  In other words, admissions decisions are not simply sequential, serial binary decisions, but rather a series of complex, inter-related ones.  Race can be a factor, although almost always a very minor one, and often comingled with other variables that don’t fall under strict construction (socio-economic status, first-generation college student status, geographic diversity, etc.) This is, by most accounts, as much in the interest of creating a community with as wide a range of backgrounds and interests as is possible, as in the interest of attempting to account for the undeniable effects of race on educational opportunity and its covariance with other factors indicated above.

There it is.  I’d love to hear what you think, especially if you disagree.