Excerpts from Review 3

Peter’s is a book to nurture free thinkers in a complex and treacherous world. A wise mind knowing the power of subverted objectivities and subjectivities, he carefully avoids definitive statements (I love his recurrent paragraphs “Should we or should not we”), but he definitely deliberates the crucial risks and benefits to align one’s vision to ranked intuitions and ordinal approximations of reality provided by others.
Like foreword author Scott Page, I found the book relatable, personable, likable and humorous. The book reflects wonderfully Peter’s rich life: it is culturally broad, has the earnest curiosity of a child wrapped in the mastery of an influential scholar. It also shows no hesitation for the author to poke fun at himself, which helps create a strong bond with the reader. It covers all the main bases of its problem, -sociology, psychology, economy, cognitive science, neuroscience, computer science and mathematics-, but woven so discreetly in
the background of an entertaining narrative that the reader might never know (s)he has absorbed a hefty dose of hard science. With all massive events in global politics showing the power of social games of influences, it is also a very timely contribution to the intellectual discourse.

1 . Overall, do the organization and progression of the book make sense? How could the structure be improved?
I found the narrative flow to be very adequate. From a bird’s eye view, the epilogue chapter 9 has a slightly uneven structure, but the reading remains fluid and pleasant throughout.
2 . Are the core arguments clear? Are they convincing and well-supported? Are these insights original? What would help?
Judging from all the areas where I have expertise, (because the book is written in a casual style that eschews the hyperformalism of academic writing), the core arguments are all solidly anchored in science, and reflect the original thinking of its author.
3 . Is there anything related to the scholarship of this work that we, outside the field, might not notice as a potential problem? Are the references up to date? Is the author engaged in the most current conversation on the topic?
The conversation is absolutely current.
4 . Are there any glaring omissions? Is any material superfluous?
No glaring omission and a quite complete overview of the problem, though if I may, I would like to suggest the following:
-a flowchart illustrating the main steps in the process of ranking would be very valuable, perhaps p.16.
-the book could have a description of Steven’s 1946 magnus opus “On the theory of scales of measurement”, perhaps p.97 in “the tyranny of metrics”. In particular, Stevens theory suggests why some ordinal violations might be encountered: rank of ranks, or composite measures are not the same due to discrepancy between ordinal and interval data.
-AI seem unescapable nowadays, as an approach to complexity reduction. Perhaps a
word on it is needed? Also progress in AI and Mathematics might allow us to reverse engineer the obscure ratings? And interestingly, mathematically discover the anomalies just like statisticians can tell corrupted voting in elections or fake statistics in scientific papers?
-I became thirsty for a longer section that would view rankings, not as a static construct, but as a nonlinear dynamical system, with its sudden phase transitions, reorganizations, changes. And some light on when/how those sudden change come about. Loss of trust in a system of ranking. Novelty. Etc. There is some of it in a few places, including p. 121 with self-fulfilling prophecies and at the beginning of the epilogue for the nonlinear part of it. In term of layperson’s experience, the mess of airfare online purchase, with online
trackers that up the price to return visitors so as to game the consumer interest Vs price ranking, would be worth contextualizing.
-I view ranking as a spatial cognitive operation. I suspect a literature on eye tracking could come in support. And then, the intricacies with memory and the hippocampus become clearer. I wonder if a neurobiological discussion on this issue could be had.
5 . Who would find this book useful and interesting? Whom do you see as the
primary readership?
I suspect that this book could reach a broad audience. It does have a multidisciplinary composition. But foremost, it is written in a very approachable manner, and in our new world obsessed with social media (and the ordinal order that it encourages), I anticipate that this book could become a darling both within and far remote from the academic circles that have seen Peter’s work thus far.
6 . What is your “bottom line” assessment of the project? Aside from copyediting, is the manuscript ready to be published in its current state? What are your specific recommendations for the author?
Some minor improvements could make the book publication-ready. It reads very nicely, is terrific in style, and is timely. I loved every bit of it.
Notes and other comments: …

[Review 2: excerpts]

This is a masterful extended essay spanning so many topics and ideas, yet in an accessible and readable manner. Kudos to the author!

Responses to OUP questions:

1. Overall, do the organization and progression of the book make sense? How could the structure be improved?
The organization is quite sensible, with each chapter building on the earlier ones.
2. Are the core arguments clear? Are they convincing and well-supported? Are these insights original? What would help?
The core arguments are clear, and most assuredly this book is very much needed. We rank everything, with little understanding of what we are doing. This book explains what is happening step-by-step. It shows the underlying bases for our seeming need to rank stuff and lays out the human dynamics that produce this need. That makes the book both focused AND intellectually sweeping. A real tour de force.
Because it is both focused and intellectually sweeping, I have one major suggestion. So many issues are addressed, and all are relevant to the book, but I think the author could more explicitly draw us back to where he is going—that is, show the reader the relevance of the material to the fundamental ideas I’m getting—lists, rankings, ratings, and competing criteria for them. Kind of several very brief internal summaries (that is, internal to the chapters) and pointers toward where we are going. I’ve tried to suggest in my specific suggestions where and sometimes how this might be done. As I’ll say now and will reiterate later, the book could be published as is and be a fine piece of work. But as Erdi would say, any of our work can be improved.

One device to do this is the Lessons Learned section at the end of chapter 2—great idea.
But it is not followed through in subsequent chapters. Maybe the author should consider adding for each chapter? Maybe Chapter Takeaways instead of Lessons Learned—the latter sounds too professorial.
Also I wonder if the reader needs a little explicit nudging toward where the book is going. For example, surely comparisons are central to rankings, and Chapter 2 does a fine job of showing case-by-case how these comparisons are subjective (and selective on the criteria we use for the rank). But the connection between comparisons and ranks is maybe not as clear as it could be. Repetition is the key to learning, and reputation that refers back to a point made earlier when an example of it is used later is particularly effective. Academic citations are (probably) a power distribution, a form of skewed distribution, so why not remind the reader of the earlier discussion? Then one gets an “aha” reaction—now I get it!
3. Is there anything related to the scholarship of this work that we, outside the field, might not notice as a potential problem? Are the references up to date? Is the author engaged in the most current conversation on the topic?
Sure seems so to me. I’ve noted a couple of scholars in my own field who might be mentioned, but certainly nothing major.
4. Are there any glaring omissions? Is any material superfluous?
There are no glaring omissions to my knowledge, and the author is a master of bringing in only the relevant material.
5. Who would find this book useful and interesting? Whom do you see as the primary readership?
This book will definitely appeal to a general audience, and should be marketed that way.
6. What is your “bottom line” assessment of the project? Aside from copyediting, is the manuscript ready to be published in its current state? What are your specific recommendations for the author?
Yes, I heartily recommend publication. It’s ready, but as I noted above, it can be even stronger with a modest level of effort.
Specific comments: XXXXXX

[Review 1]

1. To begin. The book is brilliant! brilliant! brilliant! I am not kidding you. This book will sell to a wide market if it is kept at a good price and also marketed well.
2. I really enjoyed how the author mixed stories of growing up in Budapest and otherpersonal stuff in order to illustrate the key points he is trying to make.
I was able to read the book almost in one sitting. It flows very well, the writing is very good, and while the points being made are rigorously scientific, they were easy to follow for a general readership, similar to other books on complexity or networks or books like the tipping point, etc.
3. Which leads to my main point: this is actually an incredible sociological study of the big data socio-cybernetic globalised world in which we presently live, and how the virtual world of social media has become so infused into our existence, that it has even
overtaken science and academia. I could not stop reading. ranking, ranking, and more ranking. i did not realise how essential it is to globalised life today!
As such, I think this book could be also marketed to undegraduate and graduate students in sociology and globalisation studies.
4. The other thing the author did very well was overturn the arrogance and over-confidence of so many of our complexity colleagues who pretend like the new science cannot be fooled by its own numbers and therefore can show us the pathway to success.
As such, I entirely agree with Scot Page’s review at the beginning of the book, who is a good colleague of mine and one of the top-tier leaders in the field and that will help to market the book and also establish further Peter’s credibility, which does not need really any, as he is already highly regarded!
5. In terms of competition, this book could go up against Barabasi’s new book, The Formula, which is limited in its inability to see how success does not work according to a formula. In other words, the current book is a brilliant counterpoint to that book, which is already being highly praised. And it could be marketed as such!
6. All of which takes me to the next point: cognitive bias. The chapters dealing with cognitive bias and group think and bounded rationality and the author’s usage philosophy and cognitive science to demonstrate the underlying psychology of ranking was also excellent!

This also connects the book to recent publishing trends in behavioural economics and cognitive science.

7.  I also liked how he showed readers how science actually works, and the immense difficulty surrounding the politics of academic knowledge.
8. Also, it was very clear the author put a lot of time and care into the writing. The writing is very good and also, while it was personal, it was not glib or contrived. You kept to the ideas, which was great.
9. I have no real criticisms. The only thing I noticed here and there were some typos, but that will not be an issue once the editors clean up things.

From the OUP Editor

I promised you some feedback early in the week, and here it is! First, I really enjoyed reading your manuscript…

Couple of picky points.  In a few cases, in trying to be conversational with the reader, you go a little overboard with a parenthetical comment or two. This is okay for email, but too many of these undermines your authority as an author and scientist. So just be on the lookout for these, which I highlighted. In other cases, you pitch your tone and argument too low and it comes off patronizing: for example, “Science is our friend.” Or,  “Ranking is fun.”  I always tell authors that the Oxford audience watches PBS and listens to NPR. They want to improve themselves when they willingly pick up a book….

 

I learned a lot, but the array of recent books available on the topic of ranking, not to speak of the plentiful material available online on the subject, suggests the question: Why should someone read your book?  How is it different from other similar books?  In the final chapters, your message is that we need ranking and there really isn’t anything better, but just be cautious.  How do we do this?  Can you create a list or short chapter that addresses the actions we can take to proceed cautiously?  There are many writers who are telling us that this game we all play should be abolished since it’s the wisdom of the crowds and we are stupid to be hoodwinked.
Tell us why, with the proper precautions, ranking and rating not only won’t go away, but how it’s good for us.  You need to weave in a serious argument into the many entertaining stories you tell to elevate your book to something that at times sounds like a long (albeit entertaining) magazine article and into a book-length discussion on a topic that we should take seriously.  Are there things that are suspicious that we need to be aware of on the internet?  You know so much. Share your pearls and perils with us!

One of the more interesting chapters is on reputation.  Can you make clear what we can and cannot do to game the system?  What are some actions everyone should do?  I don’t use Facebook and someone told me to have a picture and profile on LinkedIn, and so I did. But I don’t pay any attention to these kinds of things. Should everyone check out certain things online about themselves?