You can choose something (somebody) if you know it (her/him): recommendation systems

Algorithms: friends or foes?

You have not made recently any decision without seeing the opinion of the web. As I open Amazon, I see a holiday toy list, with Star Wars Droid Inventor Kit on the top. I consulted Tripadvisor when I returned to Liverpool after decades to find a small hotel near Liverpool John Moores University where I actually talked with the same title that the present book has. I don’t really use Yelp, it might be might my fault. I have my favorite restaurants in Budapest, from Spinoza to Pozsonyi Kisvendéglő. In Manhattan, do you need recommendation? Actually I learned just this summer that some – maybe mostly Italian – restaurants don’t take cash only to save credit card transactions. leads the dating websites, and Jdates is fifteenths now. (You will not believe it, but it is true. Just I wrote one more paragraph, and had to return to here since a this moment TripAdvisor sent an email with the Subject: The United States’  #1 restaurant announced! I don’t tell you who is #1, but you saw a picture at the top of this post. Fine-dining lover New Yorkers might identify the restaurant.

I am teaching a class this winter about the Complexity or Ranking, and plan to discuss with my students their experience with Netflix, so I suppose to collect nice stories, but I don’t know now the details. What we know already now of course that ”Netflix developed and maintains an extensive personalized video-recommendation system based on ratings and reviews by its customers”.

Recommendation systems use algorithms, so we do what the algorithms dictate us. The modern recommendation systems combine several strategies, by answering such kinds of questions:

  • Show me stuffs what my friends like (collaborative filtering)
  • Show me stuffs what I liked in the past (content-based filtering)
  •  Show me stuffs what fits to   my needs: (knowledge-based recommendation).

A little data science will be explained, but will not be painful. A lot of data is being collected, not only, but preferentially via social media, about consumption habits, in case of Netflix specifically about movies and TV shows, first to movies are characterized by some important features. How ”similar” are the two movies can be answered by analyzing the similarities between features. As Xavier Amarian, who served as Research Director for Netflix writes:
”We know what you played, searched for, or rated, as well as the time, date, and device. We even track user interactions such as browsing or scrolling behavior. All that data is fed into several algorithms, each optimized for a different purpose. In a broad sense, most of our algorithms are based on the assumption that similar viewing patterns represent similar user tastes. We can use the behavior of similar users to infer your preferences.” If you know the distances i.e. the dissimilarity between any two items, you can make an ordered list.

The other side of the story: Netflix addiction
I have to admit that I am not a Netflix subscriber, so I have second-hand information only. “Binge watching” is an action to watch multiple episodes of a television series in rapid succession. While it shows some correlation to depression and loneliness, more or less we understand how our brain forces us to be addict. Episodes of series end with an exciting scene, trigger is pulled, but we don’t know the implications. Such kinds of clickhangers activate stress by increasing a stress-related hormone… so you push the button, look the next episode and so on… after several hours binge watching you may have a feeling, oh it was an achievement, so your brain releases more dopamine, a substance related to award, and there is a reinforcement signal creating a self-amplifying loop. So you might spend the whole weekend by looking Netflix.


The manipulation of objectivity: an excellent demonstration

In a fine paper published today in the New York Times Amy Qin explained to the Readers of the journal how the struggle for scientific reputation drives scientists, this time from China, to publish fake research. Quantitative measures, specifically  impact factors  play the main role in career promotions. (What I believe is much better than promotion based on political loyalty).  “In June, Sichuan Agricultural University in Ya’an awarded a group of researchers about $2 million in funding after members got a paper published in the academic journal Cell.”. Why not? Cell has an impact factor 30, and I would like to believe that a journal with such a well-deserved long-term reputation still has a reliable peer review system. (We, editors of journals with much lower impact know very well how difficult to find reliable reviewers).

I think the key paragraph of the paper is this: “In America, if you purposely falsify data, then your career in academia is over,” Professor Zhang said. “But in China, the cost of cheating is very low. They won’t fire you. You might not get promoted immediately, but once people forget, then you might have a chance to move up.” The bad news is that fraud techniques are more and more sophisticated, but the banning from the participation in the scientific game for 99 years might have some repulsive power.

I might be too optimistic…

The “best” and the “most” mania

It is time now to discuss our love to ordered lists. BBC E-cyclopedia defines listmania as ”media obsession to categorise anything into lists, be they musical artistes, memorable sporting moments, quotations, words of the year etc etc”. A list of reasons are given why our brains and mind love lists. Believe or not (why not?) I spent six minutes to collect the list here starting from cnn and use at most one click:

  • 8 best Istanbul hotels
  • 5 ways you’re losing money without even realizing it
  • 7 best places to stay in Napa Valley
  • 12 amazing hotels perfect for animal lovers
  • The best photos from the solar eclipse
  • 10 of the best beaches near airports
  • 8 tips for surviving long flights
  • 4 questions to ask yourself before retiring .

Our brain process external information perceived by all of our sensory systems. The incoming information is useful only if we are able to comprehend, and lists help to organize new information. Many of us prepare To-Do lists. It is a prioritized lists of all the tasks that we need to carry out generally ”soon”. So, first we make a list of everything that we have to do, than make a ranked list with the most important tasks at the top of the list, and the least important tasks at the bottom. Actually it is not so simple to prepare a To-Do list, and the question is whether we have some ”best” algorithm of constructing one. There are different features of tasks we have to do it, say urgency, expected penalty for postponing, the time to should assign to do the job, etc. You certainly can not postpone to pick up your kid from the kindergarten. If your boss asks you to tell your quick opinion about a situation (maybe in a form of list) at noon, you will decide whether you do it before or after lunch (well, an eager beaver could do instead of lunch). Some people believe that a LONG To-Do list is the proof of their value and indispensability. Not speaking about the sad fact that cemeteries are full with indispensable people, successful people are able to outsource their tasks, as most famously Tam Sawyer did with the whitewashing of the fence.



The tragedy of being the SECOND best

”Feeling second best is probably the worst feeling in the world.” The Filipina American author Melissa de la Cruz wrote ”I’m tired of being your best friend. I’m tired of being second best. I won’t settle for that anymore. It’s all or nothing, Schuyler. You have to decide. Him or me.”

I can’t help it, I should go back to soccer, discussing the final of the soccer world cup in 1954, when Germany (that time West Germany won 3:2 against Hungary. While the West German football team’s World Cup win was a real turning point in post-war German history. The aftermath in Hungary is a particular illustration of the effect of a sport event in a poor country for politics.

Children bestseller writer (and attorney; it might be an excellent combination) Rachel Renée Russell in an article in Dork Diaries ”When you feel like you’re always second best” kick around the feelings of middle school girls:”I feel like I’m always second best. I’m always the backup friend, the third wheel. When my teachers tell us to get into pairs, I’m always the one left out. All my friends partner up, and I’m left standing there awkwardly. I’m sick of being everyone’s second choice. No matter how hard I try, I’ll never be good enough. Please help me!” RRR is ready with a possible answer ”So, what if instead of waiting, you picked someone yourself? What if instead of looking dejected, you plastered a big old grin on your face, walked right up to someone before she could choose someone else, and said, “Want to pair up?!?”” Not only middle school girls have this problem: ”Being second is to be the first of the ones who lose” as the arguably the most influential Formula One driver Ayrton Senna (1960-1994).

Abel Kiviat was the 1,500m silver medalist in the 1912 Olympics in Stockholm. He supposed to win when Arnold Jackson ”came from nowhere” to beat him by a mere one tenth of a second. About 70 ((!!) years later, at age 91, Kiviat admitted in an interview: ”I wake up sometimes and say: ‘What the heck happened to me?’ It’s like a nightmare.”

The classic studies in psychology on Olympic medalists clearly state that silver medalists tend to be miserable because they’re comparing themselves to the gold medalists; bronze medalists, on the other hand, are comparing their outcome to those who came in fourth and beyond, and so they tend to be more pleased with themselves than the silver medalists — even though the silver-winners technically beat them. The men’s epee individual competition in the Olympic in Rio de Janiero in 2016 is analyzed. I will tell details of the story in the book itself.

and the winner ….

The English author Kazuo Ishiguro has been named winner of the 2017 Nobel prize in literature, praised by the Swedish Academy for his “novels of great emotional force”, which it said had “uncovered the abyss beneath our illusory sense of connection with the world”.

With names including Margaret Atwood, Ngugi Wa Thiong’o and Haruki Murakami leading the odds at the bookmakers, Ishiguro was a surprise choice. ( The Guardian )

Who will win this year the Nobel prize in literature?

It is difficult to deny that Nobel prize can be identified with being ranked as #1.  Two days before the winner will be announced journalists are writing nice articles about the chances.  Chances – odds using the terminology  of betting – reflects  the “wisdom of crowd”, well, if you believe a gambler is wise… Historically Ladbrokes is known as a main betting firm founded in 1886.  Articles published today   in New Republic and  in The Guardian are based on the list of Ladbrokes.  The odds of the first three are:

Ngugi Wa Thiong’o 4/1
Haruki Murakami 5/1
Margaret Atwood 6/1.
Well, this are the so-called objective numbers…  Let’s wait two more days.


The Best & Worst Places for Expats in 2016

As you already know I am spending some reasonable (?) time to uncover the applied  methodologies behind different ranking procedures. Here is a  survey about ranking countries from the perspective of expats.  I am not sure why, 67 countries are listed only. The long survey describes the methods and results. Here I copy the main findings only:


  • Taiwan named best expat destination in the world
  • Malta pushes Mexico off the podium
  • Taiwan and Malta perform well in all areas of expat life
  • Ecuador loses ground in terms of Working Abroad and Quality of Life
  • Kuwait, Greece and Nigeria remain last on the list