Professor Jeff Soverns conducted a study that observed 1072 laptop users during 60 sessions of six law school courses. Here are some of his findings:

•More than half the upper-year students seen using laptops employed them for non-class purposes more than half the time, raising serious questions about how much they learned from class. By contrast, first-semester Civil Procedure students used laptops for non-class purposes far less: only 4% used laptops for non-class purposes more than half the time while 44% were never distracted by laptops.

•Students in exam courses were more likely to tune out when classmates asked and professors responded to questions and less likely to tune out when a rule was discussed or textual material read in class.

•For first-semester students, policy discussions generated the highest level of distraction while displaying a PowerPoint slide which was not later posted on the web elicited the lowest level.

•With some exceptions, what was happening in the class did not affect whether upper-year students tuned out or paid attention.

• The format used to convey information – lecture, calling on students, or class discussion – seemed to make little difference to the level of attention.

•Student attentiveness to the facts of cases is comparable to their overall attention levels.

The article speculates that student decisions on whether to pay attention are responses to the tension between incentives and temptation. While the temptation to tune out probably remains constant, ebbs and flows in incentives may cause students to resist or yield to that temptation. Because first-semester grades have more of an impact on job prospects, first-semester students have a greater incentive than upper-year students to attend to classes. Similarly, because students probably anticipate that rules are more likely to be tested on exams, students perceive that they have more of an incentive to pay attention when rules are discussed. Conversely, students may suspect that matters asked about by classmates are less likely to be tested on and so their grades are unlikely to be affected if they miss the question and answer, reducing the incentive to pay attention.

Because of methodological limits to the study, the article notes that its conclusions cannot be considered definitive, and so it urges others to conduct similar studies.

How did the Professor monitor what people were doing on their laptops?

Consequently, in the fall of 2010, I stationed observers at the back of six law school classes in an attempt to determine, among other things, the extent to which laptops distract students, and whether student use of laptops for non-class purposes is triggered by what is happening inside the classroom—for example, whether students are more likely to visit Facebook when the professor is lecturing or another student asks a question.

For each session, observers were instructed to record the number of laptops they could see, the number of students who never used the laptop for a non-class purpose (“not distracted”), the number who used them for a non-class purpose for up to five minutes (“occasionally distracted”); the number who used them for more than five minutes but less than half the class (“distracted”); and the number who used them for at least half the class (“strongly distracted”). The five-minute threshold is somewhat arbitrary but it seems a reasonable cutoff between quick checks of email over the course of a class and using laptops for something more. Observers also recorded what was happening in the class in two broad categories: content (e.g., procedural posture of a case, holding of a case) and format (e.g., was the professor calling on a student, lecturing, taking questions).

This survey differs in methodology from a different study, that gave student’s a survey, and asked them to self-evaluate their laptop usage. There are a number of methodological problems here, which the author concedes.

First, I have some reservations about the categories of not distracted, occasionally distracted, distracted, and strongly distracted are arbitrary—just because someone is looking at a laptop screen does not translate to distraction. I’m sure there are students who can gaze at a professor for 2 hours and not imbibe a single word, yet according to this rubric, he would be considered “not distracted.” Conversely, a supertasker (such as myself), who can be on his laptop the entire class, while participating, would be dubbed “strongly distracted.” There seems to be an correlation/causation issue here with respect to laptop usage for non-class purposes and distraction.

The author addresses that point in the section titled “methodological” problems:

Third, student laptop use is a crude measure of how attentive students are. Because the observers were seated behind the students, the observers often could not see where the students were looking. It is entirely possible that students who had a web site on their screen were in fact paying attention. Indeed, one observer reported that one student frequently used his laptop for non-class purposes, but would raise his hand and speak or ask questions, suggesting an awareness of what the professor had just said. Conversely, just because a student does not have a solitaire game up does not mean the student is paying attention.

This methodological problem seems quite significant.