The quality of subjective performance evaluation is dependent on the incentive structures evaluators face. Figure skating competitions provide a unique opportunity to study subjective evaluation. Using scoring data from World Figure Skating Championships between 2001 and 2003, I test for the existence of \outlier aversion" in which subjective evaluators avoid submitting outlying judgments. I find that judges manipulate scores to achieve a targeted level of agreement with the other judges. Agreement may not be a good criterion for the validity of an evaluation system, consistent with the recent applied psychology and management literature.
© Copyright 2007 All rights reserved.