Rating graduate students applicants

December. As a college professor, my seasonal duty is writing letters of recommendations, and rate students based on  several criteria to help them to be accepted by graduate schools. Students should ask a number of professors to evaluate them. Occasionally I have to tell a student that I would not be able to be a strong recommender, so it is better not to ask me. We, evaluators, combine quasi-objective data (say, grades) and subjective impressions to generate a rating score. Subjectivity is far from being identical to random, and college professors don’t have better ways to help students and graduate programs to find a good match.  Admission committee has a strong interest in ensuring they only accept mature, polite, reliable and stable people into their program, and my professional duty is to help them.

CollegeNET  is a corporation, which provides software as a service for many universities, among others for admissions and application evaluation. There are six criteria to rate students:

• Knowledge in chosen field
• Motivation and perseverance toward goals
• Ability to work independently
• Ability to express thoughts in speech and writing
• Ability/potential for college teaching
• Ability to plan and conduct research

We should choose among five options: Exceptional (Upper 5%) Outstanding (Next 15%) Very Good (Next 15%) Good (Next 15%)  (Next 50.)

(In some other softwares the “exceptional” is the upper 2%. I noticed that while I am ready to place students in exceptional category if is defined as upper 5%, and very infrequently, if they should be in the upper 2%.)

How  do we generate the numbers and choose the appropriate rubric? In principle, a micro-rationalist, bottom up approach would work: teachers could collect and store data from students back to decades, and they might have a formal algorithm to calculate the percentages. I do believe, still many of us adopts top down strategies. I ask myself: do I want to grant an “all exceptional” set of grades? Does the applicant  have a clearly weakest point, so  should I check the third or maybe the fourth rubric?  How about to check four exceptional and two outstanding rubrics?
Good or bad, decision makers calculate the sum of the grades, analyze the grade distribution.

As Churchill could have told: Quantification is the worst form of evaluation, except for all the others.