A few weeks into her sophomore year of college, Leigh Burrell got a notification that made her stomach drop.
She had received a zero on an assignment worth 15% of her final grade in a required writing course. In a brief note, her professor explained that he believed she had outsourced the composition of her paper — a mock cover letter — to an artificial intelligence chatbot.
“My heart just freaking stops,” said Burrell, 23, a computer science major at the University of Houston-Downtown.
But Burrell’s submission was not, in fact, the instantaneous output of a chatbot. According to Google Docs editing history that was reviewed by The New York Times, she had drafted and revised the assignment over the course of two days. It was flagged anyway by a service offered by the plagiarism-detection company Turnitin that aims to identify text generated by artificial intelligence.
Panicked, Burrell appealed the decision. Her grade was restored after she sent a 15-page PDF of time-stamped screenshots and notes from her writing process to the chair of her English department.
Still, the episode made her painfully aware of the hazards of being a student — even an honest one — in an academic landscape distorted by AI cheating.
Generative AI tools including ChatGPT are reshaping education for the students who use them to cut corners. According to a Pew Research survey conducted last year, 26% of teenagers said they had used ChatGPT for schoolwork, double the rate of the previous year. Student use of AI chatbots to compose essays and solve coding problems has sent teachers scrambling for solutions.
But the specter of AI misuse, and the imperfect systems used to root it out, may also be affecting students who are following the rules. In interviews, high school, college and graduate students described persistent anxiety about being accused of using AI on work they had completed themselves — and facing potentially devastating academic consequences.
In response, many students have imposed methods of self-surveillance that they say feel more like self-preservation. Some record their screens for hours at a time as they do their schoolwork. Others make a point of composing class papers using only word processors that track their keystrokes closely enough to produce a detailed edit history.
The next time Burrell had to submit an assignment for the class in which she had been accused of using AI, she uploaded a 93-minute YouTube video documenting her writing process. It was annoying, she said, but necessary for her peace of mind.
“I was so frustrated and paranoid that my grade was going to suffer because of something I didn’t do,” she said.
These students’ fears are borne out by research reported in The Washington Post and Bloomberg Businessweek indicating that AI-detection software, a booming industry in recent years, often misidentifies work as AI-generated.
A new study of a dozen AI-detection services by researchers at the University of Maryland found that they had erroneously flagged human-written text as AI-generated about 6.8% of the time, on average.
“At least from our analysis, current detectors are not ready to be used in practice in schools to detect AI plagiarism,” said Soheil Feizi, an author of the paper and an associate professor of computer science at Maryland.
Turnitin, which was not included in the analysis, said in 2023 that its software mistakenly flagged human-written sentences about 4% of the time. A detection program from OpenAI that had a 9% false-positive rate, according to the company, was discontinued after six months.
Turnitin did not respond to requests for comment for this article, but has said that its scores should not be used as the sole determinant of AI misuse.
“We cannot mitigate the risk of false positives completely given the nature of AI writing and analysis, so, it is important that educators use the AI score to start a meaningful and impactful dialogue with their students in such instances,” Annie Chechitelli, Turnitin’s chief product officer, wrote in a 2023 blog post.
Some students are mobilizing against the use of AI-detection tools, arguing that the risk of penalizing innocent students is too great. More than 1,000 people have signed an online petition started by Kelsey Auman last month, one of the first of its kind, that calls on the University at Buffalo in New York to disable its AI-detection service.
A month before her graduation from the university’s master of public health program, Auman was told by a professor that three of her assignments had been flagged by Turnitin. She reached out to other members of the 20-person course, and five told her that they had received similar messages, she recalled in a recent interview. Two said that their graduations had been delayed.
Auman, 29, was terrified she would not graduate. She had finished her undergraduate studies well before ChatGPT arrived on campuses, and it had never occurred to her to stockpile evidence in case she was accused of cheating using generative AI.
“You just assume that if you do your work, you’re going to be fine — until you aren’t,” she said.
Auman said she was concerned that AI-detection software would punish students whose writing fell outside “algorithmic norms” for reasons that had nothing to do with artificial intelligence. In a 2023 study, Stanford University researchers found that AI-detection services were more likely to misclassify the work of students who were not native English speakers. (Turnitin has disputed those findings.)
After Auman met with her professor and exchanged lengthy emails with the school’s Office of Academic Integrity, she was notified that she would graduate as planned, without any charges of academic dishonesty.
“I’m just really glad I’m graduating,” she said. “I can’t imagine living in this feeling of fear for the rest of my academic career.”
John Della Contrada, a spokesperson for the University at Buffalo, said that the school was not considering discontinuing its use of Turnitin’s AI-detection service in response to the petition.
“To ensure fairness, the university does not rely solely on AI-detection software when adjudicating cases of alleged academic dishonesty,” he wrote in an email, adding that the university guaranteed due process for accused students, a right to appeal and remediation options for first-time offenders. (Burrell’s school, the University of Houston-Downtown, warns faculty members that plagiarism detectors including Turnitin “are inconsistent and can easily be misused,” but still makes them available.)
Other schools have determined that detection software is more trouble than it is worth: The University of California, Berkeley; Vanderbilt; and Georgetown have all cited reliability concerns in their decisions to disable Turnitin’s AI-detection feature.
“While we recognize that AI detection may give some instructors peace of mind, we’ve noticed that overreliance on technology can damage a student-and-instructor relationship more than it can help it,” Jenae Cohn, executive director of the Center for Teaching and Learning at UC Berkeley, wrote in an email.
Sydney Gill, an 18-year-old high school senior in San Francisco, said she appreciated that teachers were in an extremely difficult position when it came to navigating an academic environment jumbled by AI. She added that she had second-guessed her writing ever since an essay she entered in a writing competition in late 2023 was wrongly marked as AI generated.
That anxiety persisted as she wrote college application essays this year. “I don’t want to say it’s life-changing, but it definitely altered how I approach all of my writing in the future,” she said.
In 2023, Kathryn Mayo, a professor at Cosumnes River College in Sacramento, California, started using AI-detection tools from Copyleaks and Scribbr on essays from students in her photo-history and photo-theory classes. She was relieved, at first, to find what she hoped would be a straightforward fix in a complex and frustrating moment for teachers.
Then she ran some of her own writing through the service, and was notified that it had been partly generated using AI. “I was so embarrassed,” she said.
She has since changed some of her assignment prompts to make them more personal, which she hopes will make them harder for students to outsource to ChatGPT. She tries to engage any student whom she seriously suspects of misusing AI in a gentle conversation about the writing process.
Sometimes students sheepishly admit to cheating, she said. Other times, they just drop the class.
Generative AI tools including ChatGPT are reshaping education for the students. (Representative Image) 
Share
