The Subjective Side of Ranking Colleges

The college ranking results from WSJ or U.S News & World Report can boost the demand for a college. So, now that WSJ says it is tweaking its metrics, we should take a look.

WSJ’s New Criteria

In 2022, WSJ explained that engagement, environment, resources, and outcomes determined their college ranking order. Emphasizing the first three, WSJ said that input was how to judge a school. Consequently, they quantified instructional spending, student opinions of their teachers, diversity, and peer reputation. With the outcomes component at 40 percent, the top five schools were Harvard, Stanford, MIT, Yale, and Duke.

In its 2024 version, WSJ moved outcomes up to 70 percent of the total with graduation rates and graduates’ salaries far outweighing instructional quality and diversity. Saying they are determining “value added,” WSJ gives a price tag to the number of years at school by comparing them to the graduate’s salary.

Whereas Princeton had been #8, it took the top slot in 2024:

Our Bottom Line: College Ranking Accuracy

Saying its past metrics insufficiently recognized the value they added to a student’s success, WSJ (supposedly) solved the problem with updated metrics.

I wonder, though, if we are dealing with false precision. Whether looking at colleges or best dressed movie stars or GDP, ranking has to be totally subjective. It all depends on the variables you select, their weight, how you quantify them, your time frame, and much more.

At every decision-making level, there is no way to avoid subjectivity. At the top, we can debate the criteria that were selected. Moving downward, we can ask about how to calculate the criteria for those categories. For example, for salary, which numbers do you use?  Then, moving to the commodity being judged, we can wonder about the variables.

I do not want to say that ranking statistics lie. We just need to be aware of their limitations.

My sources and more: Having just learned that WSJ switched its ranking criteria, I decided it was time to return to the problems with rankings. First though it made sense to recheck their prior indicators and compare them to the 2024 criteria.

Please note that several of today’s sentences were in a past econlife post.