My exams consist of two issue-spotting fact patterns, each with five questions, for a total of ten questions. When I first started teaching, I would grade a student’s paper straight through. That is, I would read her answers to the first fact pattern, and then her answers to the second fact pattern. After a year or so, I found that approach unsatisfactory. I worried that my scores on the second fact pattern would be impacted by my scores on the the first. That is, if a student did really well on the first question, she would score higher than she should on the second question. If she did not do well on the first question, she would score lower on the second question than she should.
This phenomenon is known as the “anchoring heuristic.” By one definition:
The Anchoring Heuristic, also know as focalism, refers to the human tendency to accept and rely on, the first piece of information received before making a decision. That first piece of information is the anchor and sets the tone for everything that follows. For example, a car dealer might suggest a price for a car and the customer will try to negotiate down from that price, even if the price suggested is more than the Blue Book Value.
I then switched my practice. I would begin grading a students’ answers to the five questions in the first fact pattern straight through. Then, I would grade all of the students’ answers to the second fact pattern, without being able to see the scores for the prior fact pattern. That approach worked well enough for a few years, but then I had more concerns. There was still intra-part anchoring, as Part I, Question 1, would impact Part I, Questions 2-5. Further, it led to unequal grading. I may grade Question #1 for Student #1 on Monday, and Question #1 for Student #80 on Friday. It is tough to claim parity when you grade the same question so far apart in time. Even with a rubric, my memory may fade for what I’m looking for.
This semester I tried something new to eliminate anchoring and promote parity as much as possible. First, to eliminate inter-Part, and intra-Part anchoring, I graded a new question each day. That is, on Monday I would grade Part 1, Question #1 for all of my students straight through. That way, I can credibly say that each question was graded within a short span of time, with an equal mindset of what I was looking for. I would then grade Part I, Question #2 on the next day, without referencing the previous score. And so one. But I added a new wrinkle. I found that when I first started grading a question, I wasn’t precisely sure what the students would discuss, so I would err on the side of safety for the first few papers, and grade higher-than-average. The difference was statistically negligible, but I was aware of it, and it bothered me. So as to avoid that benefit inuring only to the people with the lowest exam numbers (when I traditionally started), I would begin at a different exam spot for each question. For example, with Question #1, I would begin with Exam #1; for Question #2, I would begin with Exam #10; for Question #3, I would begin with Exam #20; and so on. Once I finished grading the five questions in Part I, I would then repeat the process with Part II, blindly cycling through each question separately.
I am confident this latest approach is my fairest yet. As a way to double-check the integrity of my own grading, the average score of Part I and Part II was only .9 apart. In other words, students performed consistently the same on both parts. In a few instances, some students scored +/- five or six points. I would double-check those, and in almost every case–barring some who could not finish on time and left stuff blank–a student did not know the topic in one question as much as another. Here, using blind grading as a cudgel against anchoring ensured that I did not implicitly reward a student who did well on Part I, or punish a student who did poorly on Part I. Each question was graded independently for each student.
The downside to this approach is that it took much more time. Because I planned to grade one question per day, given two exams with ten questions each, it took about 20 days to complete the grading. (I had about 80 students per class). Though it took more time, I am confident it is fairer.