Welcome to the site of Ranking

I decided to write a non-fiction book with the title and subtitle RANKING – The reality, illusion and manipulation of objectivity.  The book discusses the Hows and Whys of our love and fear of making ranks and being ranked through many real life examples to be viewed from three different angles (reality, illusion and manipulation) of objectivity. Ranking converts scientific theories to everyday’s experience by raising and answering such question as:

  • Are college ranking lists objective?
  • How to rank and rate states based on their fragility, corruption or even happiness?
  • How to find the most relevant web pages?
  • How to rank employees?

Life and society is really complex, consequently our message is not so simple such as ”Ranking is good!” or ”Ranking is bad!”. Since we permanently rank ourselves and others and are also being ranked, the message is twofold: how to prepare the possible most objective ranking and how to accept that ranking does  not necessarily reflects our real values and achievements. The reader will understand our difficulties to navigate between objective and subjective and gets help to identify and modify her place in real and virtual communities by combining our human intelligence with computational techniques.

 

A question from Peter Bruck:

Two of performers really stand out from the above selection: Beethoven and Michael Jackson – great music and great performances. But for my generation Bach, Mozart etc are missing.
However, in relation to Peter Erdi’s book the real question is: how to define rules which gives large weight to real, heartbreaking music – like the “Leiermann” of Franz Schubert – instead of the nice ass of blond girls (which is equally enjoyable but not as a musical accomplishment).
Please do make some hints!

Shy versus succesful

As I am thinking on marketing I read a blog Shy And Successful: 11 Reasons Why The Introvert Has The Edge 

Should I upload my close peer Brian Castellani’s mail? While I understand that shy people LESS LIKELY TO MAKE FOOLS OF THEMSELVES SOCIALLY, I take the risk to seem (and to be) fool. Here is the  (somewhat edited) mail:

1. Brilliant!  brilliant!  brilliant!  I am not kidding you.
2. I really enjoyed how you mixed stories of growing up in Budapest and other personal stuff in order to illustrate the key points you were trying to make.
3. Which leads to my main point: this is actually an incredible sociological study of the big data socio-cybernetic globalised world inwhich we presently live, and how the virtual world of social media has become so infused into our existence, that it has even overtaken science and academia.  I could not stop reading.  ranking, ranking, and more ranking.  i did not realise how essential it is to globalised life today!
4. The other thing you did very well was overturn the arrogance and over confidence of so many of our complexity colleagues who pretend like the new science cannot be fooled by its own numbers and therefore can show us the pathway to success.

[Brian, I deleted this paragraph: you criticized an immediate bestseller, and probably you are right, but I did not ask for your permission to make it open…]
5. All of which takes me to the next point:cognitive bias.  Your chapters dealing with cognitive bias and group think and bounded rationality and your usage of philosophy and cognitive science to demonstrate the underlying psychology of ranking was also excellent!
6. I also liked how you showed readers how science actually works, and the immense difficulty surrounding the politics of academic knowledge.
7. Also, it was very clear you put a lot of time and care into the
writing.  The writing is very good and also, while it was personal, it was not glib or contrived.  You kept to the ideas, which was great.
8. I have no real criticisms.  The only thing I noticed here and there were some typos, but that will not be an issue once the editors clean up things.

 

Foreword by Scott E Page

In this brilliant, wide ranging book, Péter Érdi, an award winning teacher and scholar, takes up the phenomena of rankings and ratings (I’ll get to the difference in a moment). A computational scientist, Érdi proves equally skilled as a social observer revealing deep implications of the ubiquitous rankings and ratings created by social and mainstream media. We assign far to much credibly to numerical rankings that may, at their core, be subjective impressions. Even more troubling, as we change our behavior to move up those lists, we allow ourselves to be manipulated by the rankings.

Rankings and these behavioral responses to them occur across the socio-technical landscape. To grasp the breadth of his inquiry, take a moment and leaf through the index. Here’s one sampling taken back to front: the Wong Baker pain scale, US News and World Report University Rankings, Scar (yes, from the {\em Lion King}), recommendation letters, the illusion of objectivity, the Hungarian National Soccer Team, Erdős numbers, ELO Chess Ratings, and Campbell’s Law. The A’s alone include Jane Austen, Aristotle, and Arrow’s Impossibility Theorem.

That sampling of topics only hints at the fun in store for you. Throughout, Péter ‘s fertile, lively mind is on full display. You are in for a treat. Though the book takes up technical topics, Péter’s writing manages to be bright, funny, and clear. By book’s end, many readers may contemplate catching the train to Kalamazoo or Budapest with the hopes of meeting up with Péter to learn more about preferential attachment mechanisms, bounded rationality, social neuroscience, the psychology of list making, or the applications of network statistics. For those of you who know Péter, particularly his former students, reading the book will remind you of his boundless, generous curiosity. The book like Péter, is informative, deep, thought provoking, and joyful.
The best rankings rely on objective criteria. Rankings of the tallest buildings, largest Northern Pike, and fastest motorcycles can be accepted on face value. However, even objective criteria may, when viewed under a microscope, contain elements of subjectivity. The official height of a building includes the building’s towers if they are integral to the building. The spire on the Freedom Tower in New York counts while the two antennae atop the Willis Tower in Chicago do not. The integral portion of a building lies in the eye of the beholder. And that’s where the problems start, with the inclusion of subjectivity.

Subjectivity allows us to rank as we see fit. In the movie version of Thomas Wolfe’s “The Right Stuff”, a reporter asks the astronaut Gordon Cooper, played by a young Dennis Quaid, to name the best pilot he ever saw. Cooper first rambles about pictures on a wall in a place that no longer exists and hurtling steel, all as a prelude to naming the Chuck Yeager. When Cooper realizes that the press wants a story and has little interest in the truth of the matter, he breaks into a toothy smile and says, “ who’s the best pilot, I ever saw? well, uh, your lookin’ at him.”

{\em Air and Space Magazine} would beg to differ. They do not rank Gordon Cooper in the top ten, though Yeager does rank. \’Erdi would be quick to point out that both Cooper’s ranking and the magazine’s, just like the ubiquitous rankings we find on the World Wide Web—the top ten beaches, the top eight Belgian ales, and the top seven dog breeds—-are subjective. Some person, or group of people, made up an ordering and then justified it using criteria derived after the fact. Nevertheless, these rankings confer a degree of authority—ah the power of numbers.

Yet, as Érdi shows, in most of the important cases, objective rankings are not possible. Permit me a brief foray into formalism. Formally, a ranking is a {\em complete, asymmetric, and transitive relation}. Complete means that it compares any two things. Asymmetric means that it ranks each item either above or below every other: either you like beets more than carrots or carrots more than beets. Transitive means that if A is preferred to B, and B to C, then A must also be preferred to C.
As logical as transitivity may seem, it can be violated by collections of rankings. In a Condorcet Triple, a majority rule vote of three people each of whom has transitive preferences, results in A defeating B, B defeating C, and C defeating A. Majority rule voting becomes an instantiation of the Rock–Paper–Scissors game. In other words, even if each person has a consistent ranking, that in no way implies that a collective ranking exists.

A similar problem arises if the items we wish to rank possess multiple dimensions. Magazines rank restaurants by evaluating the quality of their food, their ambiance, and the professionalism of their staff. They then assign numbers to each restaurant on each dimension and sum them to produce a rating. Out of a total score of 30, one restaurant may score 28, while another scores 27. These numbers, as \’Erdi points out, are subjective. One person’s five out of five may be another person’s four out of five. What appears scientific is largely made up.

In fairness, often the scores are a mix of objective and subjective. Such is the case in the US News and World Report’s college rankings which take into account the number of classes a college offers with fewer than nineteen students and the overall faculty to student ratio (both objective) as well as ranking by deans (subjective). To create ratings (which can then be simplified to a ranking), US News then attaches a weight to each of these criteria. How do they come up with the weights? Again, those are just made up based on common sense. So once again, what appears scientific is in fact subjective.

An immediate consequence of this method is that a college can improve its ranking by limiting enrollments in some classes to only nineteen students. Doing so improves the school’s ranking, Keep in mind that no empirical evidence supports a significant loss in learning from adding a twentieth student. US News just choose the number nineteen. To see the pernicious effects of just this one ranking criterion go to almost any college web page and you will see that they advertise the number of classes with nineteen or fewer students. Colleges prevent students from taking classes (sorry, you’re number twenty) so as not to hurt their rankings.

It follows, paradoxically, that even well intentioned attempts to identify the best of us may bring out the worst in some of us, as we distort ourselves to improve our rankings. Thus, the greater importance we attach to these mostly subjective rankings, the more we produce behavioral distortions. With this book, Professor \’Erdi has done us a service. He has taught us to think more deeply, yet done so with a captivating examples and a light hand.
Scott E Page
Leonid Hurwicz Collegiate Professor of Complex Systems, Political Science, and Economics, University of Michigan-Ann Arbor. External Faculty, The Santa Fe Institute