Surveillance U

The Rise of Virtual Proctoring in the Pandemic

Attending virtual classes — and taking virtual tests — isn’t ideal for anyone. All through the pandemic, faculty and students alike have struggled to create the genuine connections and rich dialogues that happen when everyone’s in a room together. Technical challenges range from random computer glitches to missing out on important bits of the lecture when the WiFi fades. Zoom fatigue spares no one. And with everything on the computer, professors need new tools to make sure their learning environments are fair, which requires an ability for those who use testing to prevent and catch cheating.

Some university administrators believed they’d found a solution to this last concern by contracting with virtual proctoring companies like, say, ExamSoft, which was founded in 1998. Since then, it has administered 75,000,000 exams, according to the company website.

Jack Schutte, a first-year student at UC Santa Cruz, shared his experience over text with YR Media during his first time taking an exam with ProctorU. (Photo courtesy of Zoe Harwood.)

ExamSoft isn’t the only such product on the market. In May 2020, Proctorio's Chief Executive Mike Olsen told the New York Times that after campuses started going remote because of the pandemic, his virtual proctoring company grew 900 percent. Another large-scale virtual proctoring service is ProctorU, which reported proctoring over 5,000,000 exams. Jarrod Morgan, founder and chief strategy officer of ProctorU, told YR Media: “We feel very strongly that we should protect the education industry and the integrity of exams.”

“The pandemic put everyone in a situation where they had to use this,” Morgan said of virtual proctoring. “Nothing really changed about our service, but the fact that we were not really an option in 2020: we were a requirement to continue to work.”

I’m a freshman virtually attending classes at University of California at Santa Cruz, and I’ve heard lots of stories from students using proctoring software for tests. By and large, the chatter amongst my friends hasn’t been good. I spent several months looking into the rise of various virtual proctoring companies, the effects they’re having on students, and whether, in the eyes of students, the benefits outweigh the costs.

Students I spoke with reported that using proctoring software added significant anxiety to the already-stressful situation of taking a high-stakes test, and that the anxiety could affect their performance. Many also expressed concern that these companies violated their rights in one way or another. Some raised issues related to digital privacy, others about racial and other forms of bias within the technology. Our data analysis of 79 Change.org petitions against virtual proctoring software in the United States echoed these themes. Petitions were launched across 28 states. Privacy — mentioned in 72 of the 79 petitions — was the issue cited the most. Over half of the petitions raised concerns that virtual proctoring drove inequality and damaged mental health:

How Virtual Proctoring Works

ExamSoft

ProctorU

ProctorU combines automated and live proctoring. The company collects biometric data and then uses third-party tools to verify student identities and track keystrokes and other data points during test-taking, according to ProctorU's 2019 Technical Security Overview. But that’s just the automated aspect of its service. With ProctorU, real, actual people sit down with you (well, via your screen, not in the room) and watch you take your test. While the proctors monitor students through their webcams, the students YR Media interviewed said that they couldn’t see the proctors at all, and could only talk to a disembodied voice. ProctorU founder Morgan said live proctors can make active interventions. For example, if a test-taker is “constantly looking out the window,” Morgan said, the proctor can remind the student: “’Hey, I need you to keep your focus here.’ It could be [the student] picking up their cell phone or could be something as flagrant as picking up an answer key.”

ProctorU’s statistics on academic integrity, from February 2021. Out of all the exams they proctored, only .72% involved confirmed “breaches.” As far as we can tell, ProctorU has removed this graphic from its website.

Respondus

Bias and other concerns related to AI are nothing new, as YR Media’s own newsroom has reported through stories featuring leading voices in AI, explorations of deepfakes in music and beyond, and an interactive investigation into facial recognition systems.

We know that tech giants including IBM, Microsoft, and Amazon all have a track record for creating artificial intelligence that carries a gender and racial bias, according to Joy Buolamwini, an AI researcher working at MIT who has studied this phenomenon extensively. Buolamwini found that because faces used to test the software often leave out women and people of color, the software is only “optimized for a small portion of the world.”

When YR asked if ExamSoft had difficulty registering faces of students of color, Nici Sandberg, the company’s associate director of marketing content and communications, said that ExamSoft "does not have any data that supports the claim of any significant problems with the software detecting faces of students of color."

But our interviews with students show that Sergine Beaubrun was not alone in her struggle to be seen. Areeb Been Khan also had trouble with his mock bar exam. He told YR Media that it took five business days before Examsoft detected his face at all. “That was anxiety inducing and it made it difficult to study information that was going to impact my livelihood,” Been Khan said.

Been Khan gave YR Media the emails he received from ExamSoft. The emails suggested he could lose his registration status to take the exam. (Photo courtesy of Areeb Been Khan.)

Been Khan said he emailed ExamSoft on a Monday, got a written response on a Wednesday, and when that wasn’t sufficient, he called customer support Friday morning but couldn’t get through because of a long wait. After selecting the option of an automated call back, he said he finally received help Friday evening.

Been Khan said that when customer service called him back, an associate made changes that eventually permitted him to take the test. “He said straight up … sometimes our program just needs, like, an updated data set … like a baseline for your face,’ which proves to me that they didn’t have a baseline for my face,” Been Khan said. “There's racism in technology, and that's kind of just what it comes down to, at least for me.”

ExamSoft confirmed to YR Media that it was aware of Been Kahn’s experience and wrote that after he called the ExamSoft support team, “his issue was resolved within minutes of speaking with a representative.”

As more allegations similar to Been Khan’s and Beaubrun’s came to light during the pandemic, the U.S. government took notice. Six U.S. Senators wrote a letter to Sebastian Vos, the Chief Executive Officer of ExamSoft, in December 2020. The lawmakers asked ExamSoft to outline plans that safeguard the civil rights of students of color and students with disabilities. The letter cites numerous media reports of proctoring apps erroneously flagging students for possible cheating.

ExamSoft responded to the senators’ letter more or less by saying: that facial recognition technology that we use... it isn’t ours. It denied that there is any bias in its products and said that the more common difficulties for exam-takers are internet connectivity, poor picture quality, and lighting issues. Responding to concerns about how the software could unfairly flag students with disabilities, ExamSoft said it was not involved with granting accommodations and that it’s up to institutions to make those accommodations when using the software.

The ExamSoft letter signed by Vos said, “We have investigated whether there have been any systemic issues with ExamID due to individual exam-takers’ race or gender identification. It would be a key priority to resolve any such issues, but we found none.”

Vos' letter added that ExamSoft was making "Deferred ID" mandatory for a majority of its institutions in 2021. The "Deferred ID" feature allows exam takers to start their exam even if there is a problem with photo matching, said Sandberg, the ExamSoft spokesperson. She did not confirm whether making the feature mandatory was a response to stories like Sergine's or Areeb's.

Jack Schutte watched with stunned amazement as his mouse and keyboard moved all on their own — or not quite on their own, but guided by an invisible proctor on the other end of his computer.

Schutte, like me, attends UCSC. He used ProctorU to take an exam for his calculus class. He and I are close friends.

The first thing Schutte had to do before he could take his test was download a ProctorU extension on Chrome. Schutte became uneasy as soon as he read the extension’s permissions, which included the ability during the test for ProctorU to “read and change all your data on the websites you visit,” “read and modify data you copy and paste,” “detect your physical location,” and “capture the content of your screen.”

“Between taking control of someone’s computer [and] reading and accessing all of your data ... it feels like someone [could] exploit the university’s use of ProctorU for nefarious ends,” Schutte said.

Because Schutte was using a product that included live proctoring, unlike Beaubrun and Been Khan, he could actually talk to the person who had control over his entire computer during the exam time. When the proctor asked for permission to take control, Schutte said he felt nervous.

Having an “outside proctor to monitor you, in your own living space... if ... there was an actual person that came and sat in your home and watched you take your test ... You feel like they're expecting you to cheat. They just want someone there to catch it when it happens,” Schutte said. But he went along with it anyway. “It was my exams,” he said. “I couldn’t exactly refuse at that point.”

UCSC covered the cost of Schutte’s exam, but that’s not the case at all universities. The University of Illinois Urbana-Champaign, for example, listed ProctorU costing between $16 and $38 per exam, depending on length, with late fees that could run between $8 and $12 if you don’t reserve your ProctorU appointment at least three days in advance. Jaylyn Stubbs, a teaching assistant from Louisiana State University, expressed concern for low-income students already struggling to stay afloat. “It's kind of unfair,” Stubbs said. “For the students that can't really afford it, I don't feel like it's fair that they have to come out of pocket to take a test.”

When asked for comment on Stubb’s concern, LSU told YR Media that faculty members are required to communicate costs to students in syllabuses at the beginning of the semester, and that the university offers “other primary assessment options.” Professors opt for ProctorU when “they feel that this is the best and only virtual route to assess the student during these times,” Alison Lee Satake, the Associate Director of Research at Louisiana State University, wrote in an email.

Universities that have added or extended use of these services during the pandemic have also done so at a cost — literally. Take my own university. In the years prior to the pandemic, UCSC’s agreements with ProctorU were largely with the university’s off-campus UC Extension programming, based on purchase orders YR Media obtained through public record requests. Between 2016-2019, these records showed the university approved $67,600 in fees to ProctorU.

When the coronavirus hit, UCSC’s Vice Provost for Academic Affairs Herbie Lee authorized spending on ProctorU for the main campus during the 2020-21 school year, and for a lot more money: $200,000.

Lee couldn’t tell us which part of UCSC’s budget was used to cover the software costs and said that he could not directly provide invoices from ProctorU that would confirm how much UCSC has spent to date on the software. “There was a lot of allocation that was done very quickly in response to the conversion to emergency remote instruction. A lot of things happen behind the scenes,” Lee said. “I wasn’t a part of exactly which type of funding was used for this.”

YR Media Journalist Zoe Harwood placed a FOIA request asking all UCs and CSUs for their vendor contracts with virtual proctoring companies. UC Santa Cruz had the largest purchase order of all the universities that responded.

Compared with some other universities in the UC system, UCSC’s $200,000 in authorized spending on ProctorU was a lot of money. UC Merced authorized spending of $20,997 on Respondus, and UC Santa Barbara, which is larger than UCSC, authorized spending around $60,000 on ProctorU from March 2020 to June 2021, according to public records requests.

Given the problems with proctoring software, plus the anxiety it can cause, I began to wonder: what if some of that $200,000 had gone towards programs that better provide mental health services to students struggling during the pandemic? Based on YR Media’s prior reporting for NPR, we know that even before the coronavirus hit, universities were grappling with how to provide vital mental health services to students. During the pandemic, students experienced mental health issues at even higher rates. With limited budgets, administrators are forced to make tough choices about what services to pay for. We can learn a lot about their priorities from where they put their money.

ProctorU wouldn’t confirm how much money it made during the pandemic. But CSO and Founder Jarrod Morgan maintained that the company provides a valuable service to education. “If we don't or somebody doesn't … make sure that ... the students that are getting the credit are the ones that did the work,” he said, “then over time, it really waters down the value of the education credential.”

Andrea Kohlenberg felt ProctorU violated her rights — and she’s taking the company to court.

Before she took her LSAT — the required test for admission to law school — Kohlenberg said she was totally fine with the idea of being proctored. “It makes sense in theory,” she said. “These are major exams. Cheating would be really bad academically.” But as Kohlenberg learned some of the things ProctorU would require from her, she got increasingly worried.

“My initial concern was security on my laptop. If they can take control on my computer, I have sensitive data on my computer. That would be serious if someone else got that. I have medical information and documents. I didn’t know if those would be safe.” said Kohlenberg.

Kohlenberg lives in Illinois, where the Biometric Information Privacy Act (BIPA) was designed to regulate the storage and purchase of data.

According to BIPA, any company that collects any biometric information must do three things: protect that data using a “reasonable standard of care within the private entity’s industry” while they still need it, destroy any biometric data once it’s no longer useful to the company, and provide a timetable to the public for when the data will be destroyed.

Kohlenberg and two other students filed a class action complaint that demanded a jury trial, alleging that the company failed to do all three of those things.

“It’s one thing if you get your Social Security Number stolen, you can get your social number changed. But if someone steals your biometrics, those are things you can’t change about yourself,” Kohlenberg said.

YR Media reached out to ProctorU for its response to Kohlenberg’s allegations and other details in this story and did not hear back by deadline.

Kohlenberg’s story is echoed by two other students we talked to. Talha Sarfraz Ahmad and Mohamed Amine Bouftas are trying to bring about change at Purdue University by using the power of student government to push back against virtual proctoring services.

Ahmad was spurred to action by an experience he had with Respondus. He said that when he first tried to use it, Respondus locked him out of his iPad for eight hours. Because all of his notes were on that device, he said he had to rush to the student library to print them out in time for his exam. YR found that out of 79 petitions on Change.org, 33 petitions made complaints against Respondus.

Ahmad was infuriated. With Bouftas’ help, they drafted a resolution against virtual proctoring services at Purdue.

Ahmad said, “I would argue that using these ... built a lot of suspicion between professors and students. And when so many people fall into problems like being falsely flagged for cheating or things like that, it makes me wonder, like, is this really achieving the objective of education?”

When asked to comment on Ahmad’s experience, Respondus said students can “restart the iPad” to leave “Assessment Mode” or the device will automatically do so after 8 hours. Ahmad told YR Media that a pop-up notification said the iPad would be locked for 8 hours.

Students aren’t the only ones taking a stand against virtual proctoring services. Some colleges discourage it. The Academic Senate at San Francisco State University (SFSU) passed a resolution that makes a case to restrict or ban the use of third-party proctoring. “Third-party remote proctoring is expressly designed to minimize cheating and rarely provides any student support,” the resolution states. “This casts faculty in the role of police catching cheaters, rather [than] faculty acting as mentors to facilitate student success.”

SFSU Professor Dr. Jackson Wilson, the vice chair of the academic senate at SFSU, helped author the SFSU resolution after hearing multiple concerns from students and faculty. He described virtual proctoring software as “a troublesome service … that we should be highly suspicious of.”

However, Andrew Roderick, assistant vice president of academic technology at SFSU, said the resolution is non-binding and is not policy. He knew of one program that used proctoring software on its culminating exam required for graduation.

Roderick said in an email, “Based on their experience using remote proctoring software, they are unlikely to use it in the future as it presented a number of logistical and technical challenges to the group of students participating in the exam.”

The resolutions passed by student governments and faculty are usually not obligatory, and universities make exceptions to use virtual third-party proctoring for certain programs. California State University Office of the Chancellor wrote a memorandum asking campus leaders to address concerns with virtual proctoring but stopped short of banning the software. UCSB Faculty Association made a similar gesture by writing a letter asking the university to curb the use of ProctorU (which the university appears to have ignored).

Filing a petition online is another popular tactic for making a statement against virtual proctoring software, and sometimes it works. The City University of New York’s petition, “Do Not Let CUNY Violate Student Privacy,” gained more than 28,000 signatures. In an update to the petition on Change.org, Aharon Grama, the chief of staff of CUNY’s undergraduate government, wrote that he is fairly certain that the university discontinued the use of virtual proctoring software after experiencing difficulties with ProctorTrack. CUNY did not return YR Media’s request for comment.

We’ve seen the impacts these virtual proctoring services are having on students now, during the pandemic. As the vaccine rollout continues and the country starts to reopen, what can we expect from proctoring apps in the future?

UCSC Vice Provost Herbie Lee said his institution rolled funding left over for ProctorU on its $200,000 purchase order into the 2020-21 academic year, authorizing continued funding for the software.

YR uncovered new patents for other virtual proctoring services, including one pending patent that works by positioning a camera directly behind students, watching over them as they take an exam. Another pending patent filed in June 2020 outlined plans for headgear that looks like Google Glass that students would wear so the device could monitor them during a test.

Screenshot from a 2020 pending patent filed by David S. Breed and Wendell C. Johnson for an invention designed to ensure that a test-taker isn’t getting help or otherwise cheating.

ProctorU Founder Jarrod Morgan said at the end of the day, it's up to schools to decide whether they want to discipline students when irregularities are flagged. ProctorU just provides the proctors and the recordings.

So what are some of the alternatives out there? What are students asking for?

Bouftas, who helped Ahmad draft a resolution against virtual proctoring at Purdue, suggested less invasive alternatives, such as virtual conferencing services already being used. “Another thing we tried to address is modifying the way assessments are given, so that cheating is harder to do,” Bouftas said.

Mohamad Bouftas and Talha Ahmad continue to discuss the bill they passed with Purdue’s senate, sparking campuswide awareness of proctoring services and the impact they can have on students. Sergine Beaubrun completed the bar exam and is now working towards becoming a criminal lawyer and prosecutor focused on fighting injustice. Meanwhile, Andrea Kohlenberg remains a class action plaintiff and representative for others suing ProctorU over their experience with the LSAT.

“We're all in this together,” Beaubrun said. “Let's fight for a better system.”