Reputation and Ranking systems

Gloria Origgi is an Italian philosopher and cognitive scientist working now at the famous EHESS (CNRS) in Paris. Her book Reputation: What It Is and Why It Matters has been published now by the Princeton University Press .

A compelling exploration of how reputation affects every aspect of contemporary life

Reputation touches almost everything, guiding our behavior and choices in countless ways. But it is also shrouded in mystery. Why is it so powerful when the criteria by which people and things are defined as good or bad often appear to be arbitrary? Why do we care so much about how others see us that we may even do irrational and harmful things to try to influence their opinion? In this engaging book, Gloria Origgi draws on philosophy, social psychology, sociology, economics, literature, and history to offer an illuminating account of an important yet oddly neglected subject.

Origgi examines the influence of the Internet and social media, as well as the countless ranking systems that characterize modern society and contribute to the creation of formal and informal reputations in our social relations, in business, in politics, in academia, and even in wine. She highlights the importance of reputation to the effective functioning of the economy and e-commerce. Origgi also discusses the existential significance of our obsession with reputation, concluding that an awareness of the relationship between our reputation and our actions empowers us to better understand who we are and why we do what we do.

Compellingly written and filled with surprising insights, Reputation pins down an elusive subject that affects everyone.”

I hope to have the book in my mail box when I will be back from Budapest to Kalamazoo,  Michigan.

Rating graduate students applicants

December. As a college professor, my seasonal duty is writing letters of recommendations, and rate students based on  several criteria to help them to be accepted by graduate schools. Students should ask a number of professors to evaluate them. Occasionally I have to tell a student that I would not be able to be a strong recommender, so it is better not to ask me. We, evaluators, combine quasi-objective data (say, grades) and subjective impressions to generate a rating score. Subjectivity is far from being identical to random, and college professors don’t have better ways to help students and graduate programs to find a good match.  Admission committee has a strong interest in ensuring they only accept mature, polite, reliable and stable people into their program, and my professional duty is to help them.

CollegeNET  is a corporation, which provides software as a service for many universities, among others for admissions and application evaluation. There are six criteria to rate students:

  • Knowledge in chosen field
  • Motivation and perseverance toward goals
  • Ability to work independently
  • Ability to express thoughts in speech and writing
  • Ability/potential for college teaching
  • Ability to plan and conduct research

We should choose among five options: Exceptional (Upper 5%) Outstanding (Next 15%) Very Good (Next 15%) Good (Next 15%)  (Next 50.)

(In some other softwares the “exceptional” is the upper 2%. I noticed that while I am ready to place students in exceptional category if is defined as upper 5%, and very infrequently, if they should be in the upper 2%.)

How  do we generate the numbers and choose the appropriate rubric? In principle, a micro-rationalist, bottom up approach would work: teachers could collect and store data from students back to decades, and they might have a formal algorithm to calculate the percentages. I do believe, still many of us adopts top down strategies. I ask myself: do I want to grant an “all exceptional” set of grades? Does the applicant  have a clearly weakest point, so  should I check the third or maybe the fourth rubric?  How about to check four exceptional and two outstanding rubrics?
Good or bad, decision makers calculate the sum of the grades, analyze the grade distribution.

As Churchill could have told: Quantification is the worst form of evaluation, except for all the others.

A news about the world of ranking: Napoleon was the Best General Ever, and the Math Proves it.

Ethan Arsht published an analysis of Ranking Every* General in the History of Warfare. He adopted  a “system of Wins Above Replacement (WAR). WAR is often used as an estimate of a baseball player’s contributions to his team. It calculates the total wins added (or subtracted) by the player compared to a replacement-level player. For example, a baseball player with 5 WAR contributed 5 additional wins to his team, compared to the average contributions of a high-level minor league player. WAR is far from perfect, but provides a way to compare players based on one statistic.”

Arsht constructed a database from Wikipedia. The database  includes 3,580 unique battles and 6619 generals, and using a not too complicated linear model he got some remarkable results:

“Among all generals, Napoleon had the highest WAR (16.679) by a large margin. In fact, the next highest performer, Julius Caesar (7.445 WAR), had less than half the WAR accumulated by Napoleon across his battles. Napoleon benefited from the large number of battles in which he led forces. Among his 43 listed battles, he won 38 and lost only 5. Napoleon overcame difficult odds in 17 of his victories, and commanded at a disadvantage in all 5 of his losses. No other general came close to Napoleon in total battles. While Napoleon commanded forces in 43 battles, the next most prolific general was Robert E. Lee, with 27 battles (the average battle count was 1.5). Napoleon’s large battle count allowed him more opportunities to demonstrate his tactical prowess. Alexander the Great, despite winning all 9 of his battles, accumulated fewer WAR largely because of his shorter and less prolific career….”

For the details visit his website, see also comments on his post.

Comparison

People have a desire to compare themselves with others. In many cultures children learn they should win to demonstrate that they are better, stronger and more successful than the others.

Direct comparison might lead to different results from Muhammad Ali’s famous ”I am the greatest” via ”The grass is always greener on the other side of the fence.” Actually, Ali stated even more: ”I’m not the greatest. I’m the double greatest. Not only do I knock ’em out, I pick the round. I’m the boldest, the prettiest, the most superior, most scientific, most skillfullest fighter in the ring today.” In principle we may believe that self-qualification is suspicious, and leads to woolf-boolf type biased ranking. Ali’s statement on himself, however, is approved by the ”collective wisdom”: almost everyone from the generation who saw him in the ring believes that Ali was really the greatest. When the U.S. Army measured Ali’s IQ at 78, he said, ”I only said I was the greatest, not the smartest.” I find amazing how objectively describes himself: ”It’s just a job. Grass grows, birds fly, waves pound the sand. I beat people up.”

As opposed to the result of Ali’s positive self-ranking, another class of comparison expresses inferiority complex. The idea behind the quotation ”The grass is always greener on the other side of the fence” may have its origin in the poem of Ovid (43 BCE – 17 or 18 CE) Art of Love. He wrote, ”The harvest is always richer in another man’s field.” There are other proverbs expressing similar attitude: ”The apples on the other side of the wall are the sweetest,” ”Our neighbour’s hen seems a goose,” and ”Your pot broken seems better than my whole one.” These all convey the message of others have a better life, are more fortunate than we are. The German version of the proverb states that ”Kirschen in Nachbars Garten schmecken immer besser” (The cherries in the neighbour’s garden always taste better.) A life might be miserable if you always feel that others have better stuffs. The feeling makes you envy and might lead to anxiety and to other mental health problems. The suggestion of Robert Fulghum, author of former best seller book All I Really Need to Know I Learned in Kindergarten is not only more objective, but also offers a viable strategy: ”The grass is not, in fact, always greener on the other side of the fence. No, not at all. Fences have nothing to do with it. The grass is greenest where it is watered. When crossing over fences, carry water with you and tend the grass wherever you are.”

The idiom ”comparing apples and orange” refers to situations, when two items practically cannot be compared. Apples and oranges are thought incomparable or incommensurable. In many European language ”comparing apples and pears” are used. Since comparison is the basis of any ranking procedure, and it has a unique role in or decision makings, I will argue that we need to find the balance between accepting reality and make an effort to change things towards future successes.