The Subjective Side of Ranking Statistics

Among national universities, in 2021 Columbia University was ranked #2 by U.S. News & World Report:

This year, it plunged to #18.

The problem appeared to be inaccurate statistics. In 2021, Columbia’s data indicated that it spent more on instruction than Harvard, Yale, and Princeton combined. In addition, its average class size and the ratio of students to faculty were debatable. Then though, checking the numbers, a Columbia math professor challenged their accuracy. Columbia admitted it had made some mistakes.

Rather than one school’s mistakes, the real problem is ranking

Our Bottom Line: Ranking Criteria

I wonder if we are dealing with false precision. After all, one number as a metric is persuasive. However, whether looking at colleges or best dressed movie stars or GDP, ranking has to be totally subjective. It all depends on the variables you select, their weight, how you quantify them, your time frame and much more.

One example is the whopping 20 percent of the U.S. News score based on “peer assessment.” Surely thinking of how his peers will assess him, the President of N.J.’s Rowan University sends homemade hot sauce to other college CEOs. His reason, he said, is to display his school’s entrepreneurial spirit. Similarly, in one New Yorker article, Malcolm Gladwell described a Car and Driver ranking system for sports cars. Switching criteria, he showed how the #1 spot changed.

As a result, at every level, rather than that seemingly objective number, we wind up with pervasive subjectivity. At the top, we question the categories that were selected. Moving downward, we can ask about how to calculate the criteria for those categories. For example, what metric conveys a student’s engagement or a country’s happiness? Then, moving to the commodity being judged, we can wonder about the numbers the participating institutions submit.

I really do not want to say that statistics lie. We just need to be aware of their limitations.

My sources and more: For a look at ranking, a current NY Times article and one from the past at the New Yorker Magazine had the ideal synergy. Then to form your own opinion, do take a look U.S. News’s category weighting in its methodology section. Or, you could prefer to select your own criteria for ranking countries.

Please note that parts of today’s post were previously published at econlife.