Adjusted Medians

The impetus for adjusting the averages of the general items came largely from the research of Anthony Greenwald (Psychology) and Gerald Gillmore (Office of Educational Assessment) which presented evidence for a causal relationship between lenient grades and high ratings.¹ The goal of the adjustments is to statistically equate classes on reason for students taking the course, class size, and grading leniency, all of which are correlated with rating results. It should be noted that other national researchers in this area have concluded that the positive relationship between grades and ratings may be explainable by differences in learning and motivation across classes and, thus, that grading leniency may not be a source of invalidity. Certainly, the adjusted averages will not be valid for every class, but we feel they will improve accuracy when considered across all classes rated.

We use a multiple regression approach to adjust medians. Adjustments are based on enrollment reason (Item 31), class size, and relative grade (Item 23). The three corresponding variables are defined as follows:

• ER (Enrollment Reason) = Percentage of students taking the course in their major, minor, as an elective, or other (as opposed to as a program or distribution requirement);
• LS (Log of Class Size) = The natural log of class size; and
• RG (Relative Grade) = Class mean of the following item:

Relative to other college courses you have taken: Do you expect your grade in this class to be:

Much Higher			Average			Much Lower
7	6	5	4	3	2	1

Considering Item 1 (The course as a whole) as an example, the simple correlations with the three variables above, based on over 25,000 classes rated during the past three years, are:

Enrollment Reason (ER)	.18
Log of Class Size (LS)	-.23
Relative Grade (RG)	.36

Entering these three variables as predictor variables, the regression equation explains 20% of the total variance (i. e., R² = .20) in Item 1. The equation for the adjusted median is:

median1 – [2.487 + (.003292 * ER) – (.143 * LS) + (.337 * RG) – 3.8829]

The correlation between the adjusted and unadjusted averages is .89. Similar equations have been developed for Items 2 through 4 and the combined average of Items 1 through 4.

An Example. Table 1 and Figures 1, 2 and 3 show the amount by which the original median for Item 1 (The course as a whole) is adjusted under various conditions. For example, if all students in your class are taking the course as a requirement of the major or an elective, 0.10 will be subtracted from the original value of the median. If your class has an enrollment of ten students, 0.14 will be subtracted. Finally, if your average rating on Item 23 (Relative Grade) is 4.5, 0.11 will be added. Taken together, a median for Item 1 of 4.55 would be adjusted as follows:

---

¹Greenwald, A.G. and Gillmore, G.M. “Grading leniency is a removable contaminant of student ratings.” American Psychologist, 53, 1997, 1209-1217.
Greenwald, A. G. and Gillmore, G. M. “No pain no gain? the importance of measuring course workload in student ratings of instruction.” Journal of Educational Psychology, 89 (4), 1997, 743-751.