The PGR aggregates the judgments of a panel of field experts into a ranking of philosophy faculties. It is desirable to give users of the guide an accurate sense of the degree of consensus among respondents about the departments ranked in the survey. In practice this means showing where departments rank but also providing information about the degree of consensus among raters. The Report has long done this numerically by reporting several measures of central tendency in the rankings (e.g., mean, median, mode), but for this edition we decided to go beyond this. 

Originally, we thought it would be useful to calculate confidence intervals (CIs) for the mean scores. These would show a range around each department's mean scores, giving some information about the precision of the estimate. However, two issues arose. First, the design of the PGR makes correctly calculating CIs a little tricky. Specifically, raters rate many departments and not all raters rate all departments. This wrinkle in the design can be accommodated by various statistical methods (which we investigated). But there is more than one way to do this calculation and we did not want the precise method chosen to become a matter of pointless controversy. Second, and more important, the PGR has long reported mean ratings rounded off to one decimal place (4.2, 3.7, etc), in part so as not to encourage invidious comparisons. To be properly informative, the CI calculation would have to be done on the raw scores, and the range reported to two decimal places. Calculating CIs with pre-rounded mean scores would have given inaccurate results. Similarly, calculating CIs on the raw scores and then subsequently rounding the means and the ranges would also give inaccurate intervals. In both cases this would defeat the purpose of calculating accurate intervals. 

In the end we decided to maintain consistency with earlier surveys and report mean scores to one decimal place, as has long been the PGR’s practice. However, we were still committed to conveying accurate and useful information on the range of rater assessments of departments. Therefore in this edition, for the first time, we are providing detailed figures showing the distribution of votes for all departments in the overall rankings. We present them as histograms and as kernel density plots. The interpretation of both kinds of figure will be explained in the “Note on the Figures” accompanying the report. We feel these visualizations convey the necessary information in a detailed and accessible way.  (Brit and I would especially like to thank Kieran Healy for his assistance in preparing the figures.)