The SAT versus Adversity Score Battle: Don’t Confuse Measurement with Decision Making! {draft 0}
UNEDITED!!!! (Natalie is in Italy). ANY COMMENTS are welcome!!
By Péter Érdi is the author of the forthcoming book “Ranking. The Unwritten Rules of the Social Game We All Play” Oxford University Press.
Stop the battle! The traditional SAT and the recently introduced Adversity Score just measures different features of a student. The decision maker’s responsibility to decide how to aggregate them to a unique score before an admission decision is being made.
Recent papers indicated the emergence of a new battle about the possible fair rules of college admission (SAT’s New ‘Adversity Score’ Will Take Students’ Hardships Into Account, By Anemona Hartocollis; The SAT’s Bogus ‘Adversity Score’Are we really going to rank students on a one-to-100 pseudoscientific index of oppression? By Thomas Chatterton William).
College Board is known to be a non-profit organization, with the mission of developing and managing SAT test to measure ”college readiness”. Traditionally it ranks students based on the evaluation of their knowledge they learned in high school, and what they need to succeed in college. Since it is justified that socioeconomic status shows a strong correlation with SAT results, College Board has a concern for many years how to take into account socioeconomic inequality influencing SAT results. In any case, two numbers seem to be more informative than one, so Adversity Score is being suggested as just an another measure characterizing a different feature, namely socioeconomic status, of student, .
There are two questions, what we should separate when we are thinking about scoring and ranking students. First, how appropriate these measures to reflect objectively the college-readiness and socio-economic status? Second, how decision makers should combine the two measures to provide a fair ranking?
First, objectivity is associated with concepts, like reality, truth, and reliability. Objectively ranking the tallest buildings in the world is relatively easy, since it is based on verifiable facts, and we have a result that everybody will accept. SAT is a number, a data based on measurements. Of course, even a very carefully fabricated test can approximate some “objective truth” only. SAT is certainly not terribly bad. It never happens that a very weak student obtains 1600-points. It is a better way of comparison than adopting teacher’s verbal characterizations.
Adversary score is one attempt to measure the socioeconomic status of a student. While I don’t know the methodological details, I am sure it is not perfect, cannot be perfect. As Thomas Chatterton Williams correctly argues in his opinion article ‘The SAT’s Bogus ‘Adversity Score’ May 17, 2019), the new measure cannot grasp all essential events of a life, which contributed to increase disadvantage. What we should understand and accept, that scores many times don’t reflect the reality, only the illusion of objectivity, still they generally work better than verbal qualifications.
Second, like it or not, social ranking is with us. Admission offices should rank our students based on a multi-dimensional scale, and knowledge is one of them, but only one of them. I was in high school in Budapest in the sixties when I learned that other things matter. Admission to college/university was a privilege and there was a very selective entrance exam. You could receive maximum 20 points for your performance, and for different majors there were different thresholds depending on the number of applications. For popular ones you had to obtain 19.5 point to ensure your acceptance. However if your dad had a Commemorative Worker-Peasant Medal (you may or may not have an idea what does it mean, but it is a different story) 14 point was sufficient to become a college student.
So, how to combine knowledge and disadvantage level? I don’t believe we have any silver bullet. However, a scoring rule with two independent variables should work better than with one. Those institution, who are worrying to see the decline in academic rigor at their college, might rely on previous knowledge/performance based acceptance. Other institutions might set their main mission to reduce social inequality, and set a higher weighting factor for Adversity Score. One might argue that schools have the right to decide about the future direction of their institutions, and determine how to weigh the different factors.
Ranking of people, schools, products, countries and just about everything else is part of our daily life. We are in a paradoxical relationship with ranking: ”ranking is good because it is informative and objective; ranking is bad because it is biased and subjective, and occasionally, even manipulated.” While there is no perfect ranking procedure, I think the inclusion of Adversity Score is step towards having better evaluation system, but we should leave freedom for the institutions to decide about their ranking based on their institutional goals.
Dr. Érdi serves as the Henry Luce Professor of Complex Systems Studies at Kalamazoo College (perdi@kzoo.edu), and the author of the forthcoming book “Ranking. The Unwritten Rules of the Social Game We All Play” Oxford University Press
The first reflection: thank you Leo H!
Flyer (at this time for friends only)
Cover
Ranking Algorithms: Application for Patent Citation Network
A book chapter written by
Ranking Algorithms: Application for Patent Citation Network
in: Information Quality in Information Fusion and Decision Making
How do technologies evolve in time? One way of answering this is by studying the US patent citation network. We begin this exploration by looking at macroscopic temporal behavior of classes of patents. Next, we quantify the influence of a patent by examining two major methods of ranking of nodes in networks: the celebrated “PageRank” and one of its extensions, reinforcement learning. A short history and a detailed explanation of the algorithms are given. We also discuss the influence of the damping factor when using PageRank on the patent network specifically in the context of rank reversal. These algorithms can be used to give more insight into the dynamics of the patent citation network. Finally, we provide a case study which combines the use of clustering algorithms with ranking algorithms to show the emergence of the opioid crisis. There is a great deal of data contained within the patent citation network. Our work enhances the usefulness of this data, which represents one of the important information quality characteristics. We do this by focusing on the structure and dynamics of the patent network, which allows us to determine the importance of individual patents without using any information about the patent except the citations to and from the patent.
“Rankings are little more than an indication…”
…of the MBA market at a particular moment. They reflect the prevailing conditions such as salaries, jobs available and the situation at a school at the time the survey was carried out. Results of rankings can be volatile, so they should be treated with caution. The various media rankings of MBA programmes all employ a different methodology. None is definitive, so our advice to prospective students is to understand the ethos behind each one before deciding whether what it is measuring is important for you.
Read more about MBA ranking methodology:
