From RANKING to REPAIR!

As you already know I am spending some reasonable (?) time to uncover the applied  methodologies behind different ranking procedures. Here is a  survey about ranking countries from the perspective of expats.  I am not sure why, 67 countries are listed only. The long survey describes the methods and results. Here I copy the main findings only:

 

• Taiwan named best expat destination in the world
• Malta pushes Mexico off the podium
• Taiwan and Malta perform well in all areas of expat life
• Ecuador loses ground in terms of Working Abroad and Quality of Life
• Kuwait, Greece and Nigeria remain last on the list

Emmanuelle Tognoli in her kind comment suggested: As we develop computational literacy in the decades to come, perhaps we will adopt “personalized rankings” just like we do of “personalized medicine”: each and everyone will be able to weight the factors (rank=30% teaching + …) and write their own equations (or have a website write it for them with sliders) to see their unique customized rankings depending on their own priorities. 

I find her suggestion that  might lead to a method to set subjectively the parameters of an objective framework. As I see, two Korean scientists (Gae-won You and Seung-won Hwang published  already ten  years ago with the title: Personalized ranking: a contextual ranking approach .  I take the liberty to copy the abstract for further common thinking:

“As data of an unprecedented scale are becoming accessible on the Web, personalization, of narrowing down the retrieval to meet the user-specific information needs, is becoming more and more critical. For instance, in the context of text retrieval, in contrast to traditional web search engines retrieving the same results for all users, major commercial search engines are starting to support personalization, improving the search quality by adapting to the user-specific retrieval contexts, e.g., prior search history or other application contexts. This paper studies how to enable such personalization in the context of structured data retrieval. In particular, we adopt context-sensitive ranking model to formalize personalization as a cost-based optimization over context-sensitive rankings collected. With this formalism, personalization is essentially retrieving the context-sensitive ranking matching the specific user’s retrieval context and generating a personalized ranking accordingly. In particular, we adopt a machine learning approach, to effectively and efficiently identify the ideal personalized ranked results for this specific user. Our empirical evaluations over real-life data validate both the effectiveness and efficiency of our framework.”

 

The 10 most beautiful universities in the US

There are many beautiful universities in the US.  As I am reading this list… how objective is the list? Of course, beauty is not necessarily objective, as we all know. How  will this list influence the number of applications next year? Do you know any other universities you would like to see on this list?

Here is a new list . If you try to understand the methodology reading public sources (as I do now), read. (Friend from CEU, you are not listed, since :universities are excluded from the World University Rankings if they do not teach undergraduates”.

“The Times Higher Education World University Rankings are the only global performance tables that judge research-intensive universities across all their core missions: teaching, research, knowledge transfer and international outlook. We use 13 carefully calibrated performance indicators to provide the most comprehensive and balanced comparisons, trusted by students, academics, university leaders, industry and governments.”

Of course, there are out of blue weighting factors. As concerns the large categories, their numerical values were set as:

Teaching (the learning environment): 30%

Research (volume, income and reputation): 30%

Citations (research influence): 30%

International outlook (staff, students, research): 7.5%

Industry income (knowledge transfer): 2.5%.

If you dig a little deeper you will see the role of reputation surveys.

Dear Reader: How do you see the adopted method from the perspective of objectivity-subjectivity?

 

 

 

 

[I take the liberty to copy the first two paragraphs of the book to illustrate my recollection to a story how a a class of boys in the elementary school answered unanimously for the question ’Who is your best friend?”]

It is not possible to play soccer if you don’t have a ball! But we had one! We played! I grew up in Budapest (well, in Pest and not in Buda, as my wife did; but I promised her not to make jokes anymore about the cultural differences in the two parts of the city) soon after the war. The elementary school had students (actually boys; no co-ed existed that time) from Angyalföld, (the now disappearing working class’ Land of Angels) and Újlipótváros (‘New Leopold Town’) particularly popular among middle-class intellectual citizens of Jewish origin. While there was an obvious social contrast in the background of our parents, (and I don’t speak here about the sad family stories hidden by their (well, our) parents from the New Leopold Town kids), the love of soccer bridged the gap. In the early nineteen-fifties Hungary had the worlds’  best soccer team led by Ferenc Puskás, whose left foot made him one of the greatest players of all time. This book is about ranking, and I think I share the opinion of many others, who believe that he was one of the two best-known Hungarians of the 20th century (Béla Bartók is arguably the other). The Hungarian team was unbeaten during thirty three games from 1950-1954 and the run ended in the 1954 World Cup final in a historical loss against West Germany (it was the first big postwar success of the new Germany). I will go back to this story later analyzing the sadness of being second best. So, soccer was extremely popular, almost all of us played almost every day for eight years.

But now we are in a classroom of forty boys. ”Who is your best friend?” – we had to write the answer for the question asked by our teacher. Thirty seven votes went to Péter Erdélyi. He had a wonderful sense of humor, but it was not the reason of his big win. His dad was a director of a state owned (what else?) company called ”Cultural articles” dealing with expensive soccer balls. Well, we lived in a poor country, so everything which manufactured was expensive. So, Péter was the only boy in the class who had a real soccer ball…

…

Please note, that a teacher asked us, and our answers were anonymous. We really were so thankful to have the chance to play with a real soccer ball, that we felt Péter was our best friend. No doubt he led the popularity list in the whole year.

 

We permanently compare ourselves with others. High school class reunions, say, are wonderful opportunities to compare our success in any aspects of life from attractiveness via career and intelligence to marriage. The self-evaluation of our own attitudes, abilities and beliefs is based on comparison with others. This observation turned to be a celebrated theory of social psychology, called social comparison theory published by Leon Festinger in 1954. We don’t necessary like to see that we are overweighted compared to our former teammates, but generally (well, I wrote.. generally, so not always…) we have the social skills to control our envy feelings. The quote attributed to former U.S. president Teddy Roosevelt, ”Comparison is the thief of joy”, still we can’t stop to compare. Social psychologists still analyze our motivation to compare, and in their book “Friend and Foe: When to Cooperate, When to Compete, and How to Succeed at Both” Adam Galinksy a social psychologist from Columbia University) and Maurice Schweitzer (from the Wharton School of Business at the University of Pennsylvania) write that ”when it comes to using social comparison to boost your own motivation, here is the key rule to keep in mind: Seek favorable comparisons if you want to feel happier, and seek unfavorable comparisons if you want to push yourself harder.” You may not be able to quit your social-comparison habit, but you can learn to make it work for you.”

Upward and downward comparison: person compares herself with other who are better/worse than her. There are from literature to pop culture, reflected also in the idiom ”Keeping up with the Joneses”. Here I add just one example from my own life:
”As a young adult I had two close friends, say John and Joe. In the seventies and early eighties people did not necessarily have a car in Budapest. If they had, most likely it was an ”Eastern” car, the most common was called Trabant produced in East Germany. It adopted two-stroke engines, what was very obsolete even that time. It was told that two people needs to its construction, one who cuts and one who glues, as it was from plastic. There was a story I remember: A donkey and a Trabant meet the Thuringian Forest. ”Hi car!” – greeted the donkey. ”Hi donkey! – answered the Trabant. ”It is not nice to call me donkey, if I addressed you as a car. You should have called me at least as a horse! I bought a six year old Trabant in my mid-thirties as my first car. It was not a status symbol, but it had four wheels.. John does not have any car (not only he could not afford it as a mathematician, but he had high diopter glasses, could not get driving licence). Textbooks suggest that the positive effects of any downward competition is gratitude what I certainly felt. While I don’t believe I felt the textbook’s negative effect (scorn), but I might have experienced some superiority. Joe, who worked for a French company soon got a ”Western” car, actually a Renault type. Did I felt any hope or inspiration, the positive effects of upward comparison of the textbooks? Probably my aspiration increased to afford (well, in a distant future a Western car. As concerns negative effects, I cannot deny I was envy. Was John unhappy or frustrated? Absolutely not! Relevance is a necessary condition of social comparison, and he was absolutely not interested in having cars! “

International Conference On Intelligent Computing Liverpool,UK August 7-10, 2017

see

Ranking: The Reality, Illusion and Manipulation of Objectivity

How to rank employees? Just to mention and example: in the last several years Yahoo’s Quarterly Performance Review has extensively been discussed not only in the business/leadership literature, but in the popular magazines, too. It is very difficult or impossible to find objective measures to promote gifted and lay off inappropriate employees. Still, we know that employees are different in terms of their ability, performance, so we should understand the challenge to rate and rank people. The link is eighteen months old, still it is worth to read.

toThere are several excellent books, which discuss different aspects of ranking, from the mathematical algorithms to ranking academic institutions, countries or political candidates.

The book   Who’s #1?: The Science of Rating and Ranking  written by the mathematicians Amy N. Langville and Carl D. Meyer (Princeton Univ. Press, 2012) grew up from the authors’ studies on mining the Web, and offer a broad overview on the mathematical algorithms and methods used to rate and rank sports teams, political candidates, products, Web pages etc., and is naturally located on a shelf for math books.

Engines of Anxiety: Academic Rankings, Reputation, and Accountability (Russel Sage, 2016) by two sociologists, Wendy Nelson Espeland and Michael Sauder analyzes the history and present practice of evaluation and ranking of the quality of higher education institution, particularly law schools. Ranking not only reflects but also shapes social hierarchy. The book well demonstrates our paradoxical emotions to ranking, quantification of performance are both needed very much and also source of anxiety.

Rather parallelly, Ranking the World: Grading States as a Tool of Global Governance   (Cambridge Univ. Press, 2015) edited by Alexander Cooley and Jack Snyder (Editors) describes our paradoxical emotions to ranking. International ranking of states performance are characterized by about hundred different indices, from ”Human Freedom Index” via ”Corruption Perceptions Index” to ”World Happiness”. The pattern is that ranking organizations are not totally independent, and some ranked countries (say, China and Russia) occasionally reacted angrily to the performance analysis, still they are interested in the result.

Majority Judgment: Measuring, Ranking, and Electing (MIT Press, 2011) by by Michel Balinski and Rida Laraki is about ranking political candidates, and the authors argue that ”The intent of this book is to show why the majority judgment is superior to any known method of voting and to any known method of judging competitions.”

I was somewhat positively surprised that the narrative of all the three social science books seems to be conform with the take away of my own planned book: we like ranking with the hope that it reflects objectivity, but frequently the objectivity is illusion only, and might be subject to manipulation. The challenge is to write a popular, easily readable integrative book on ranking and rating in accordance with both the modern theories of social and computational sciences and the everyday’s experience