Idea Block, Twitter, Traffic Lights, and Other Signals

I had a boss who once talked about “sending ideas out to the universe.”  It’s poppycock, of course, if for no other reason than the universe doesn’t care.  But that doesn’t mean it won’t work sometimes, I suppose. Yesterday, I tweeted this:

And this morning, the Twitter-verse unknowingly responded with this:

Which gave me–as only Twitter can–the ability to legally eavesdrop on three colleagues: John Lawlor, Deb Maue, and Chris Lydon.  I know exactly what the conversation means: Measuring the quality of the educational products of a college by measuring the freshman class is like measuring how good a basketball team is by evaluating the average height of its players.  It might work, on occasion, but that doesn’t mean it’s right.  However, I responded to all three, suggesting that in fact graduation rates were inputs, in a weird sort of way.

That probably would have been the end of it, but it got connected to something I frequently see on my walk between Union Station and my office, about a mile each way.  And it is especially common as it gets colder and people are eager to get indoors.

Here’s what happens: You’re standing at a corner with twenty other people waiting for the light to change, and someone notices no traffic coming, so they start to cross against the light.  Others follow, even though the sign is clearly “Do Not Walk.”  People come from the other direction, and, seeing the throngs of people crossing, assume the light says “Walk” and proceed to do so.  But by this time, a car or truck or two has come barreling down the street, and I almost witness a pedestrian hit by a vehicle.  What happened?  It’s clear that the pedestrians have equated large groups of people crossing the street with a “Walk” sign.  And usually, that signal is right; occasionally it’s not, and trouble ensues.

Thus, a blog post. And the end of Idea Block. And the head knocking resumes.

If we attempt to measure the value of an institution by its outputs (graduation rates) we are really just confusing signals, like pedestrians on Jackson Boulevard in Chicago on a cold morning. We’ve become so used to the mix up of inputs and outputs that we forget to look at the real signals; because in a way, inputs and outputs are the same things.  Don’t believe me?

Take a look at this screen shot of IPEDS Data visualized.  It shows test scores on the x-axis, and graduation rates on the y-axis. Note how they line up?

That alone would be enough to make the point, I think.  But play with this visualization by clicking here.  See if you can find any type of institution where it doesn’t hold: Urban, rural; any region; any religious affiliation;  public or private.  And notice two more things:  The color of the dot (which represents a single institution) shows the percentage of freshmen with a Pell Grant.  And the size of the dot shows the rejection rate: More selective institutions have bigger dots.  Then, just for fun, filter to institutions that don’t’ rely on student tuition to manage the budget.  Just pull the top slider down to 50%.  See who’s left.

Then ask yourself: Is a high graduation rate a function of what goes on inside the institution? Or could it be a function of selectivity, test scores and family income (which are pretty much the same thing) and resources?

Would love to hear what you think.


Life, and EM: A Series of Trade-offs.

There are very few people who understand that Enrollment Management is, at some level, an exercise in managing trade-offs.  Even though the old Michelob Light commercial suggests that you can have it all, in fact, you can’t.  And in reality, if you could, you wouldn’t be working in higher education. (Those of you who know me also know I like good German and American beer, but I’ll keep my comments about Michelob Light to myself for now.)

So, helping people understand trade-offs is a critical component of working in Enrollment Management:  If you want to push up or down on quality, quantity, diversity, or net revenue, the market is going to be more than happy to push back on you, often harder and more dramatically.  But very few people outside of Enrollment Management understand this; those who do grasp the concept of trade-offs probably don’t have the time or the inclination to dive into the details to see the nuances.

My job is essentially trying to hit a sweet spot: Managing to generate enough net revenue to pay the professors, heat the buildings, buy computers, and keep the library stocked with academic journals and important books; keeping quality as high as we can in light of the need to pay the bills; and not ever giving up on a critical component of our mission to educate those whose economic situation might not normally assume a private university education, because offering low-income students anything less than a top-quality education only adds insult to injury. Keeping these things in balance is vital to accomplishing what we set out to do.  And we re-invent the way we do it every year, because the number of students in the world is fixed, and competition is pretty fierce.  On top of it all, every university has a different recipe for success.

Historically, we’ve managed this delicate and ever-shifting balance by using SPSS and Excel to examine the relationships between and among the variables we are interested in; typically, we spend several days a year doing nothing else, and it often involves Powerpoint decks of literally hundreds of slides.  When your attention span is as short as mine, I guarantee you lose something important while day dreaming.

So, for internal use and to illuminate the balancing act, this year I took four years of data and rolled it into my favorite visualization tool, Tableau Software. In the interest of full disclosure, I’ve served on their Customer Advocacy Board, because I’m a fan, not because of the free T-shirts, or the beer I’ve been promised by my former account manager for five years now!  I have no financial interest in the company.

The data is confidential, of course, so I can only show you a screen shot, which has been sanitized by removing the values and the axis labels.  But look at this: With just a click or two, you can make a shallow or a deep dive to see the give-and-take between and among a handful of variables: Which students have the highest GPA?  How much do test scores vary by financial need?  Are men or women better students (as if we don’t already know the answer to that one!)  What percentage of our first-generation college students are from Illinois? Who helps us accomplish our mission? Who helps us pay the bills?  Which college has the most attractive students? OK, perhaps that last one is not in the data set.  But you get the idea.

If you work in EM, you owe it to yourself to explain to your campus community the ins and outs of your profession; if you work in higher ed but not in EM, you owe it to yourself to educate yourself about how these important variables relate to each other.  How you do that is up to you, but I strongly recommend against a 247-slide presentation.  You can do better.

Test-optional admissions: A year hence

If you’re a visitor to this site, you know about DePaul University’s decision to become test-optional for freshman admission.  With just a few days to go before our final, official count of freshmen (the fall census), the results from our first admissions cycle are interesting:

  • About ten percent of all applicants applied as test-optional
  • About three percent of all admits applied as test optional
  • Just under five percent of all enrolling students applied as test optional

As one faculty member put it, “So this wasn’t such a big deal, then.”  To which I politely replied, “Well, I hate to say I told you so…” (and of course, I love to say “I told you so.”)

Taking both a higher view of, and making a deeper dive into the data suggests some responses to our early critics:

  • This was a record year for applications to the freshman class
  • It will be the largest freshman class in history; we had predicted our yield to go down, but it actually stayed exactly even with last year (22.9%)
  • The average GPA of enrolling students is at its highest point ever
  • Even once we add in the test scores of the students who applied test-optional (we are collecting them for research purposes) this year’s ACT average will be at a record level (or on par with last year, if not quite a record)

In short, those people who believe a test-optional approach stains our academic reputation seem to have little evidence to back that stance up.

When you start sorting out the test-optional applicants, however, some more interesting patterns emerge:

  • Just under half were White
  • At every High School SES (Socioeconomic Status) band, the test-optional students had higher GPA’s than their counterparts who submitted test scores.  For the students from the lowest SES Schools, the average GPA was a hair under 4.0 (adjusted to fit on a 4.0 scale)
  • Their ACT Scores were as high as 29
  • 35% came from the top 1/8 of the High School Graduating Class; among low SES Schools, this number was 71%
  • Almost 60% came from the highest third of SES High Schools
  • They were distributed mostly toward the middle of our overall academic distribution: Not the superstars, but clearly not the last ones admitted to the class, either.

Now some important research begins: We’ll carefully track the progress of these students as a group, but also individually to see if they progress toward graduation in ways that other students like them (test scores, GPA, strength of curriculum, family income, first-generation status, or ethnicity) do.  Based on results at other universities who have done this, we’re confident those results will be good; if not, we’ll figure out where the gaps are and respond appropriately.

All told, this has been a liberating year after almost 30 years of doing admissions and enrollment work; doing this research, connecting with other people who feel the same, and just thinking about students in more dimensions has been quite a boost.

We’re not out to change the world, but to maybe make opportunity for kids who otherwise might not have thought it possible.  Maybe one of them will end up changing the world.  As with everything, time will tell.

Bier, but not in a Higher Education Context

Occasionally, I write about other things: Things outside the world of Higher Education. This is one of those times.

Today, I’m writing about my favorite Bier style and a Dan Fogelberg song, and in the best tradition of John Donne and the metaphysical poets, I’ll try a little discordia concors as a diversion.

I love a good Bier, even though I only drink about two a week. My favorite style is Märzen Bier, a German style known for its copper color, tan head, very smooth caramel malt flavor, and its low bitterness.  For me, that last part is especially important: Although you can’t really brew Bier without hops, a tendency in the last decade is to see just how much hop bitterness you can cram into a Bier (another is to try to make Belgian styles.  That’s a rant for another day).  Some Bier geeks like to find the hoppiest Bier they can, and lots of microbreweries are catering to them.  I just don’t like Bier with high hop profiles, but I do like this line from a New Yorker article about extreme Bier as it sums up my opinion perfectly:

Garrett Oliver, the brewmaster at Brooklyn Brewery, told me recently. “When a brewer says, ‘This has more hops in it than anything you’ve had in your life—are you man enough to drink it?,’ it’s sort of like a chef saying, ‘This stew has more salt in it than anything you’ve ever had—are you man enough to eat it?’

So, this is my favorite time of the year, in part because I love cooler weather, but mostly because Märzen Bier is now available in stores.  You probably don’t know what  Märzen Bier is, unless I call it by its more traditional name: Oktoberfest beer.  There are lots of good ones out there: Paulaner, Hacker-Pschorr, Great Lakes, Samuel Adams, and Köstritzer, to name a few, and others, like Spaten, Ayinger, and Erdinger that other people seem to like more than I do.

In the 1800’s in Germany, when Märzen became the de facto Oktoberfest style, it was seasonal because there was no refrigeration.  You can read about that here. I’m glad to say we’ve come a long way in 200 years, and you can now brew and lager Märzen all year long.  But the Oktoberfest name makes it sound like you’d only want to drink it in the fall.

It occured to me that it suffers the same fate as the Dan Fogelberg song, Same Old Lang Syne. You hear it every year at Christmas just because the song’s story line takes place on Christmas Eve. But the song is not about Christmas.  Similarly, we get Märzen in the Fall because it’s been labeled Oktoberfest Bier. But it’s delicious all year long, and I find it extremely refreshing even on a hot summer evening.  If you don’t believe me, ask Gordon Biersch, the restaurant where a waitress told me that Märzen is the biggest seller all year round.

Now, I don’t really care if I hear the Dan Fogelberg song in July.  But I’d like to get Märzen available more widely all year long.  Just stop calling it Oktoberfest Bier, and see what happens.

Who’s with me?

What Laymen and the SCOTUS Need to Know About Admissions

Note:  Although I’ve said it elsewhere on this blog, I want to emphasize that these opinions are mine.

Ever since the Michigan cases, and now with the Fisher case before the Supreme Court, something has bothered me: The way the average person just does not understand how college admission is done.  I suppose this is true of every profession: I have no idea what an accounting job or a chef’s job is like, for instance.

But just as we don’t choose Supreme Court Justices by looking at GPA and LSAT Scores and Bar Exams, neither do we make an admissions decision based on two simple criteria.  Every case in which someone says another “less qualified person” was admitted to his or her detriment misses this point completely.  And as my old logic professor used to tell us: Always attack the premise.

So, as we were thinking about joining an Amicus Brief with several other Catholic Colleges and Universities, I tried to put into writing what I’ve learned about admissions in the last 30 years.  Here goes:

  • At its core, an admissions decision suggests that someone or some group of people at the university has determined that the student is capable of doing the necessary college-level work to graduate with a bachelor’s degree.  Without this, any other discussion is moot: Admissions professionals consider it abhorrent to admit a student they deem incapable of succeeding.
  • At the same time, statistical models are really not as good as people might believe in predicting the extent of college success.  While the overwhelming majority of students admitted mange to stay above the threshold of satisfactory academic performance, there is of course wide variation between those who perform at a C (2.0 level) and those who perform at a A (4.0) level.  (And given central limit theorem, it will never be possible to enroll a class of students who all perform “above average.”) Using any combination of academic variables we collect at the point of admission, we can only get a statistical r-squared of about .2, meaning that only 20% of this variance is explained by the inputs we presume to be telling; the rest is clearly determined by other factors not deemed academic in nature.  This is true almost everywhere, not just at DePaul.
  • Thus, the definition of “Who is qualified for admission?” seems to be essentially unanswerable by the two simple things many deem to be the sole criteria: HS GPA and test scores.
  • This is especially telling when a college or university has to deny students who, based on simplistic criteria, appear to be more academically qualified than those who are admitted. When you only have so much room, you’re faced with admitting a whole group of kids who look and act alike, or mixing it up a little to improve the educational environment.
  • And additionally, such choices are compounded when an admissions officer attempts to measure potential rather than accomplishment (which of course presumes that “accomplishment” can be standardized across 35,000 high schools and hundreds of thousands of teachers offering millions of sections of classes.)  Obvious inequality of opportunity makes measuring either prior academic achievement or potential to succeed virtually impossible.
  • The history of college admission, however, is not one of making simplistic decisions based on two variables (N.B. that some state schools are mandated to do exactly this, however).  Admissions officers have long sought additional information (writing samples; letters of recommendation outlining, among other things a student’s learning styles and ability to contribute to the classroom and the community; out-of-class accomplishments) to round out the picture of the students they admit to the university, because a university is not solely an enterprise dedicated to passing information to those who are ostensibly most capable of receiving it.  John Cardinal Newman, for instance, in “The Idea of a University” and later Jaroslav Pelikan in “The Idea of the University: A Reexamination” both suggest that a university is a community that develops a wide variety of people for a wide range of outcomes.  Newman even suggests that only 1/3 of university education occurs in the classroom, suggesting that the remaining two-thirds come from self-directed study and other students.  Other factors, often far removed from academic realms (athletic ability, legacy status) are often considered in light of their effect on the total university community.
  • Beyond the core decision, however, one must consider the particular university’s mission or its raison d’etre.  Typically, the mission does not just stipulate why a particular university exists, but also articulates certain obligations, for instance:
    •  To the students it serves.  At DePaul, this ranges from traditional liberal arts programs to professional programs with career focus.  The university has an obligation to educate students who come seeking broader intellectual exposure or top-quality professional preparation, or both; all of which require exposure to a wide range of ideas and people.
    • To the community it serves. The mission statement of DePaul articulates a symbiotic relationship with Chicago, a diverse, global city, in which the university draws from, and contributes to, the city in an attempt to improve both.
    • To the market for its graduates. The critical underpinnings of financial and academic success of DePaul are based on the university’s ability to produce graduates who are ready to work in a demanding, diverse, changing society and workplace.
  • Thus, admissions officers take into account a wide variety of factors when considering an individual candidate, because admissions strives to create a campus community best aligned with the university mission.  In other words, admissions decisions are not simply sequential, serial binary decisions, but rather a series of complex, inter-related ones.  Race can be a factor, although almost always a very minor one, and often comingled with other variables that don’t fall under strict construction (socio-economic status, first-generation college student status, geographic diversity, etc.) This is, by most accounts, as much in the interest of creating a community with as wide a range of backgrounds and interests as is possible, as in the interest of attempting to account for the undeniable effects of race on educational opportunity and its covariance with other factors indicated above.

There it is.  I’d love to hear what you think, especially if you disagree.


College Admission: The Kardashian Effect

The other night my wife and I were sitting on our deck with guests and long-time friends Nancy and Al Maly from Grinnell.  Among us we have about 60 years of college admission experience, so naturally the topic tends to come up when we get together.  That, and Al’s fascination with the proper way to pour a Weißbier.

My wife suggested that I’m so enamored of my employer DePaul because of its commitment to finding talent among populations colleges don’t always serve well.  In my case, that would be first-generation and low-income.  (I remember filling out the FAF, now called the FAFSA, in 1970-something and being taken back that my father made $17,000 in his best year; mom didn’t work outside the home.)  There may be something to that.

But more important, I think, is my belief that I got a pretty good education despite the fact that I didn’t go to a name brand institution.  And a lot of people I know can say the same thing; collectively, we seem almost apologetic for not going to colleges we never even thought we could or should dream about.  And we’re doing just fine, as are many people who did go to the Holy Grail Schools of my generation.

I suggested that if students today were to dedicate as much effort to real learning and intellectual exploration as they spent on getting into college in the first place, everyone might be better off.

To which my wife replied, “Today it’s more about the wedding than the marriage.”

Brilliant.  How often to you (quietly) posit that the elaborate, over-the-top wedding portends a short marriage?  Or that the couple that keeps it simple and respectful has their heads screwed on straight?

Like a Kardashian wedding, it seems college admission today is more about the big event than it is about what comes after.  And I think that’s a shame; not on the students, but on the parents, colleges, and counselors who have made it that way.  We can do better.