YR logo Copy Created with Sketch.
Get Involved Get Involved
YR logo Copy Created with Sketch.
Get Involved Get Involved
  • News
  • Arts + Culture
  • Identity
  • Tech
  • Health
  • Opinion
  • Behind Our Masks
  • ADP.FM Stream
  • Adult ISH Podcast
  • RYL Studios
  • About YR Media
  • Apply to Classes
  • Our DIY Tools
  • Careers at YR
SurveillanceU Logo

When Virtual Proctoring Goes Wrong

As the pandemic drags on, colleges maintaining hybrid learning environments are still using virtual proctoring software. The AI-powered technology monitors students’ every move during tests to watch for signs of cheating. But how accurate is it, really? And what are its unintended effects?

After petitions against virtual proctoring launched across the U.S. and YR’s own coverage of this trend, we built a simulator to show how virtual proctoring can flag normal behavior as suspicious. This isn’t an exact replica of the software, but it’s real close. No worries, this interactive does NOT collect your data, or video; the experience runs solely on your browser.

Live Proctor

Don’t you dare look away from your computer!

What gets flagged as suspicious behavior?

Student’s gaze not focused on camera.This is a common flag virtual proctoring companies use to label students. Proctorio tracks how many times your gaze shifts away from the camera, in addition to mouse movement, and how frequently you type.

Second face detected. ProctorU says on its FAQ’s page that “allowing other people in the room” counts as a potential indicator of cheating. Better lock the door! Don’t have a private room? Have to care for young children? Too bad.

Unusual noises coming from the microphone. Pen-click at your own risk! Examsoft says it tracks abnormal behavior, like sound that could indicate an integrity breach. ProctorU constitutes “speaking aloud” as “potential aberrant behavior.” Proctorio lets teachers decide whether students with noisy housemates are cheaters.

Face undetected. People of color told YR they experienced problems getting facial detection software to see them. One student reported that she had to stand on a table — next to a ceiling light — because the virtual proctoring AI couldn’t detect her.

Subject out of frame. During a virtually proctored exam, a student resorted to peeing in a pot to avoid getting up. True story. No toilet access is a fear many students have, so much so, that Proctorio addresses it.

Pace of typing unidentifiable. Mouse movements suspicious. Companies, like Examity, say that biometric data (like keystroke speed) is unique and can be used to identify test takers. That keyboard cat walk looks like trouble! Proctorio can also see how frequently you move your mouse and ProctorU can take outright control of your mouse whenever it wants!

What are teachers saying?

What happens after you’re flagged by virtual proctoring software? The information goes to the institution that is testing you, leaving your teachers to decide if you’re guilty or not. Shea Swauger, a librarian and PhD student who has written about dismantling racism and inequity in education, says that we need to have more “serious conversations as why the f*** we’re teaching this way.” Swauger is concerned that the software is an “incredible violation of privacy” and could promote ableism, classism, racism, and transphobia. “I couldn’t design a more discriminatory system if I tried,” Swauger said.

In May 2021, YR spoke with two students of color who said they were delayed in taking a mock LSAT exam when ExamSoft technology initially failed to detect them — the exact thing Swauger fears: “Proctoring is part of a larger arc of higher education tracking things as a way of showing effectiveness and proficiency.” And even if the software does provide more data, do teachers even have enough time to sift through it?

For Swauger, part of the solution lies in increasing government oversight, and lucky for him, it’s already happening. In December 2020, six senators wrote virtual proctoring companies asking them to respond to news reports documenting instances of discriminatory bias and perceived privacy violations.

Profile picture of Shea Swauger
Shea Swauger , academic librarian and researcher at University of Colorado Denver

As a teaching assistant at Louisiana State University (LSU) in Baton Rouge, Jaylyn Stubbs also had concerns about virtual proctoring services — although hers had less to do with privacy and more to do with social class. As YR Media reported last May, LSU was one school where students had to pay for their own exams. “... fifteen dollars, is like, it might not seem like a lot of money, but that's money that nobody wants to give to a math test.”

Stubbs said that she wasn’t that concerned about privacy because proctoring was already a big part of her education, and she wanted to make sure everyone was taking the tests without cheating. “I understand the system itself, but for the students that can't really afford it, I don't feel like it's fair that they have to come out of pocket to take a test that was already in place before they even got in the class,” Stubbs said. “Some students aren't able to, you know, eat three times a day. So it’s like you're making them take three or four tests a semester. It may sound small, but that's money that they can put towards something else or that's money that they don't even have.”

Profile picture of Jaylyn Stubbs
Jaylyn Stubbs , teaching assistant at Louisiana State University

Dr. Jackson Wilson, the vice chair of the Academic Senate at San Francisco State University, says that teachers should be suspicious of the software, not the students. He outlined concerns that virtual proctoring technology “casts faculty in the role of police catching cheaters, rather [than] faculty acting as mentors to facilitate student success.”

One method teachers and students are using to organize against use of the technology is passing resolutions with their Academic Senates and student governments. That’s exactly what Dr. Wilson did at San Francisco State University, which now has a resolution that encourages educators not to use virtual proctoring. As virtual proctoring continues to grow in remote learning settings, how will it affect the trust needed to build strong communities essential for learning?

Concerns about trust are also being echoed by students, who fear the technology might negatively affect their relationships with teachers. “I would argue that using these [tools] kind of built a lot of suspicion between professors, students,” said Talha Ahmad, an undergraduate at Purdue who authored a student government resolution against the technology. “And when so many people fall into problems like being falsely flagged for cheating or things like that, it makes me wonder, is this really achieving the objective of education?”

Profile picture of Dr. Jackson Wilson
Dr. Jackson Wilson , vice chair of the Academic Senate at San Francisco State University

Explore how students are organizing

Explore Student Petitions Read YR’s investigation Tweet Your Experience

Credits

Reporter:
Zoe Harwood
Project Lead:
Jenn Cain
Producer:
Amber Ly
Development:
Radamés Ajna
Fellow:
Devin Glover
Design Coordinator:
Eli Arbreton
Design:
Marjerrie Masicat, Pedro Vega Jr.
Editor:
Lissa Soep
Chief Executive Officer:
Kyra Kyles
Director of Communications:
Angela Serna
Senior Video Producer:
Chaz Hubbard
Interns:
Ariel Tang, Dante Ruberto, Yuki Guan, Bayani Salgado, Amanda Chieng, Nubia Correa

Special thanks

Special thanks to Marshall Jackson, Sam Lavigne, Donielle Conley, Malachi Segers-Seker, Liz Tril, Luis Flores, Jalen Black, Ryan Pham, Tamara Sanchez, Cureha Mitchell

YR Media is grateful for support in this work from the National Science Foundation. The opinions, findings, and conclusions or recommendations expressed are those of the makers of SurveillanceU: When Virtual Proctoring Goes Wrong and do not necessarily reflect the views of the NSF.

This project is part of a larger collaboration with the App Inventor team at M.I.T. to make stories, apps and learning resources about A.I. through an equity lens. Stay tuned for more.

Disclaimer

We know what you're thinking: does this simulator accurately reflect the virtual proctoring experience? In fact, virtual proctoring software does NOT interrupt students taking tests to flag suspicious behavior, but it does report behaviors like the ones we’ve featured. The live virtual proctor in our simulator is actually a pre-recorded video. This interactive is intended to show how technology can mistake seemingly normal student behavior as reprehensible.

The YR SurveillanceU Simulator uses open source facial detection models published on Github. These free, off-the-shelf models differ from those used by virtual proctoring on the market, but we can’t say exactly how, because companies’ facial detection models tend to be proprietary. They are likely to be much more complex and better trained than the basic, light-weight one we’re using to maintain a playful experience that can run entirely on our users’ browsers. That said, because virtual proctoring companies don’t disclose their AI technologies, we will never fully know how accurate or biased that technology is.

This live simulator runs best on desktop browsers and modern mobile devices.

YR Media © 2021

  • About YR Media
  • Support Our Work
  • Careers at YR
  • Apply to Classes
  • Contact Us
  • Privacy Policy

Get Fresh Updates