After years of steady growth and a pandemic-related explosion, online learning has become a common format for college courses. A decade ago, just 28 percent of all U.S. college students took at least one of their classes online. By the 2021–22 school year—after widespread pandemic-related lockdowns had ended—that had doubled to 54 percent, or 10.1 million students nationwide.
This shift has helped institutions by keeping the virtual door open during emergencies, broadening their pool of potential students, and decreasing brick-and-mortar operating costs. It also has benefitted students by expanding access to advanced coursework for degree-seekers who live in remote areas or juggle coursework with other responsibilities.
We know less, however, about how online learning affects student learning. While prior research has found negative or mixed effects, these are based on self-selected groups of students. College students typically choose their institution and course of study, including whether they take some or all of their classes online.
Our study is the exception. We are, variously, officers in the United States Army and graduates of and former civilian faculty at the United States Military Academy West Point, a unique institution established to educate future Army officers. To apply, aspiring students face a demanding battery of admissions requirements, including outstanding academic and athletic records, evidence of leadership potential, and a nominating letter, typically written by a member of Congress. To graduate, they must complete academic, military, and athletic training, pass several physical fitness tests, and earn 120 college credits. Unlike other institutions, students have little control over their class schedules. More than 80 percent of the academic program is standard across all majors, and coursework is identical across instructors, who follow a unified curriculum with the same materials and graded events.
These unique attributes allowed us to conduct a randomized controlled experiment during the fall 2020 semester, when West Point brought students back to campus but also used online instruction to limit class sizes, maintain social distancing, and slow the spread of Covid-19. We compare coursework, grades, and results of a post-course survey among 551 students who were randomly assigned to online and in-person sections of Principles of Economics, a required class that most students take their sophomore year.
We find that student learning, focus, and engagement suffer when instruction moves online. Online instruction reduces a student’s final grade by 22 percent of a standard deviation, or about 1.7 percentage points—equivalent to declining to a B+ from an A–. The impact is larger for males, at 27 percent of a standard deviation, compared to 9 percent of a standard deviation for females. There is virtually no impact for Black students, while grades for white students are 28 percent of a standard deviation lower when they take the class online.
We see negative effects from online learning across the entire grade distribution but find the biggest impacts on the least academically prepared students. Compared to their in-person counterparts, grades are lower by 45 percent of a standard deviation for students with prior military service, 38 percent of a standard deviation for students who attended a stepping-stone preparation school prior to being admitted, and 27 percent of a standard deviation for students with the lowest scores on an admissions exam. We also find that online students are less likely to report feeling connected and focused compared to in-person students.
These findings do not indicate that all online learning is detrimental. The classes in our study were identical except for their setting, so students did not experience some unique attributes of some online courses, such as self-paced study, that could positively affect outcomes. In addition, because West Point classes are smaller and feature more individualized attention than the large lecture halls used in previous experiments comparing in-person and online courses, the negative impacts on instructor-student relationships may be especially large.
However, when we consider other ways that West Point differs from traditional colleges and universities, our findings are concerning. West Point students are among the nation’s most disciplined young people. They have met stringent, multidimensional requirements to enroll in a structured military leadership program that accepts about 12 percent of applicants. They attend class in uniform, maintain peak physical fitness, and commit to at least five years of active-duty military service after graduation. If online learning has negative effects at West Point, what does that mean for the typical student?
