UNEDITED!!!! (Natalie is in Italy). ANY COMMENTS are welcome!!
By Péter Érdi is the author of the forthcoming book “Ranking. The Unwritten Rules of the Social Game We All Play” Oxford University Press.
Stop the battle! The traditional SAT and the recently introduced Adversity Score just measures different features of a student. The decision maker’s responsibility to decide how to aggregate them to a unique score before an admission decision is being made.
Recent papers indicated the emergence of a new battle about the possible fair rules of college admission (SAT’s New ‘Adversity Score’ Will Take Students’ Hardships Into Account, By Anemona Hartocollis; The SAT’s Bogus ‘Adversity Score’Are we really going to rank students on a one-to-100 pseudoscientific index of oppression? By Thomas Chatterton William).
College Board is known to be a non-profit organization, with the mission of developing and managing SAT test to measure ”college readiness”. Traditionally it ranks students based on the evaluation of their knowledge they learned in high school, and what they need to succeed in college. Since it is justified that socioeconomic status shows a strong correlation with SAT results, College Board has a concern for many years how to take into account socioeconomic inequality influencing SAT results. In any case, two numbers seem to be more informative than one, so Adversity Score is being suggested as just an another measure characterizing a different feature, namely socioeconomic status, of student, .
There are two questions, what we should separate when we are thinking about scoring and ranking students. First, how appropriate these measures to reflect objectively the college-readiness and socio-economic status? Second, how decision makers should combine the two measures to provide a fair ranking?
First, objectivity is associated with concepts, like reality, truth, and reliability. Objectively ranking the tallest buildings in the world is relatively easy, since it is based on verifiable facts, and we have a result that everybody will accept. SAT is a number, a data based on measurements. Of course, even a very carefully fabricated test can approximate some “objective truth” only. SAT is certainly not terribly bad. It never happens that a very weak student obtains 1600-points. It is a better way of comparison than adopting teacher’s verbal characterizations.
Adversary score is one attempt to measure the socioeconomic status of a student. While I don’t know the methodological details, I am sure it is not perfect, cannot be perfect. As Thomas Chatterton Williams correctly argues in his opinion article ‘The SAT’s Bogus ‘Adversity Score’ May 17, 2019), the new measure cannot grasp all essential events of a life, which contributed to increase disadvantage. What we should understand and accept, that scores many times don’t reflect the reality, only the illusion of objectivity, still they generally work better than verbal qualifications.
Second, like it or not, social ranking is with us. Admission offices should rank our students based on a multi-dimensional scale, and knowledge is one of them, but only one of them. I was in high school in Budapest in the sixties when I learned that other things matter. Admission to college/university was a privilege and there was a very selective entrance exam. You could receive maximum 20 points for your performance, and for different majors there were different thresholds depending on the number of applications. For popular ones you had to obtain 19.5 point to ensure your acceptance. However if your dad had a Commemorative Worker-Peasant Medal (you may or may not have an idea what does it mean, but it is a different story) 14 point was sufficient to become a college student.
So, how to combine knowledge and disadvantage level? I don’t believe we have any silver bullet. However, a scoring rule with two independent variables should work better than with one. Those institution, who are worrying to see the decline in academic rigor at their college, might rely on previous knowledge/performance based acceptance. Other institutions might set their main mission to reduce social inequality, and set a higher weighting factor for Adversity Score. One might argue that schools have the right to decide about the future direction of their institutions, and determine how to weigh the different factors.
Ranking of people, schools, products, countries and just about everything else is part of our daily life. We are in a paradoxical relationship with ranking: ”ranking is good because it is informative and objective; ranking is bad because it is biased and subjective, and occasionally, even manipulated.” While there is no perfect ranking procedure, I think the inclusion of Adversity Score is step towards having better evaluation system, but we should leave freedom for the institutions to decide about their ranking based on their institutional goals.
Dr. Érdi serves as the Henry Luce Professor of Complex Systems Studies at Kalamazoo College (perdi@kzoo.edu), and the author of the forthcoming book “Ranking. The Unwritten Rules of the Social Game We All Play” Oxford University Press
From RANKING to REPAIR! 