Skip to main content

Brightspace Support

Brightspace Support

Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector

Posted by on Wednesday, August 16, 2023 in Announcements, News.

In April of this year, Turnitin released an update to their product that reviewed submitted papers and presented their determination of how much of a paper was written by AI. As we outlined at that time, many people had important concerns and questions about this new tool, namely how the product exactly works and how reliable the results would be. After several months of using and testing this tool, meeting with Turnitin and other AI leaders, and talking to other universities who also have access, Vanderbilt has decided to disable Turnitin’s AI detection tool for the foreseeable future. This decision was not made lightly and was made in pursuit of the best interests of our students and faculty. 

 

When Turnitin launched its AI-detection tool, there were many concerns that we had. This feature was enabled for Turnitin customers with less than 24-hour advance notice, no option at the time to disable the feature, and, most importantly, no insight into how it works. At the time of launch, Turnitin claimed that its detection tool had a 1% false positive rate (Chechitelli, 2023). To put that into context, Vanderbilt submitted 75,000 papers to Turnitin in 2022. If this AI detection tool was available then, around 750 student papers could have been incorrectly labeled as having some of it written by AI. Instances of false accusations of AI usage being leveled against students at other universities have been widely reported over the past few months, including multiple instances that involved Turnitin (Fowler, 2023; Klee, 2023). In addition to the false positive issue, AI detectors have been found to be more likely to label text written by non-native English speakers as AI-written (Myers, 2023). 

 

Additionally, there is a larger question of how Turnitin detects AI writing and if that is even possible. To date, Turnitin gives no detailed information as to how it determines if a piece of writing is AI-generated or not. The most they have said is that their tool looks for patterns common in AI writing, but they do not explain or define what those patterns are. Other companies that offer popular AI detectors have either begun to either pivot to other business models (Edwards, 2023) or closed down entirely (Coldewey, 2023). Even if other third-party software claimed higher accuracy than Turnitin, there are real privacy concerns about taking student data and entering it into a detector that is managed by a separate company with unknown privacy and data usage policies. Fundamentally, AI detection is already a very difficult task for technology to solve (if it is even possible) and this will only become harder as AI tools become more common and more advanced. Based on this, we do not believe that AI detection software is an effective tool that should be used.

 

Moving forward, with Turnitin’s AI detection tool disabled, how should instructors handle concerns about students improperly using AI writing tools? First, instructors should communicate with their students early about this. Many students are open to discussion about using AI, what is allowable or not, and how this usage can/will impact their future both in college and in the workforce after they graduate. Clearly outlining expectations and guidelines will go a long way to guiding how AI can be used in your course. If AI usage is allowed in courses, disclosures and citations for its use should be used. There are already guidelines on how to cite AI and ChatGPT to meet APA (McAdoo, 2023) and MLA (2023) citation formats. Additionally, instructors can also consider reformatting assignments to mitigate any concerns about AI usage. Options like in-class writing, requiring students to write about specific topics discussed in class, or even focusing on current issues that AI tools are not trained on are all helpful steps in revising assignments. Additionally, the Center for Teaching Excellence at the University of Kansas has a great resource on “Adapting your course to artificial intelligence” (n.d.).  One key aspect to remember throughout all of this is to balance the importance of mitigating inappropriate AI usage while also being mindful of AI’s benefits in the teaching and learning process. 

 

If you suspect that students are using AI inappropriately, there are things you can look at to determine if a submitted text is AI-generated. First, if you can, compare this writing to other work from the student. Does it match their style, tone, and level of previous writing? You can also look for inaccuracies in sources, arguments, facts, etc. AI text generators often have no concept of something being accurate or truthful, so they may often “hallucinate” and generate whole sources out of nothing (De Vynck, 2023; Sands, 2023). If you have concerns about students misusing AI, talk to them about it. Some students will just admit the use upfront if approached correctly. 

 

As we move into the future, AI tools will be more and more prevalent. AI tools are already available in places like Google Drive and Bing search, and will soon be included in everyday products like Microsoft Office. For more information on how to use AI in your teaching, please refer to the links below for various resources from across campus. 

 

References

  1. (n.d.). Adapting your course to artificial intelligence. University of Kansas, Center for Teaching Excellence. Retrieved July 15, 2023, from https://cte.ku.edu/adapting-classes-artificial-intelligence-era 
  2. (2023, March 17). How do I cite generative AI in MLA style? MLA Style Center. Retrieved June 12, 2023, from https://style.mla.org/citing-generative-ai/ 
  3. Chechitelli, A. (2023, May 23). AI writing detection update from Turnitin’s Chief Product Officer. Retrieved June 1, 2023, from https://www.turnitin.com/blog/ai-writing-detection-update-from-turnitins-chief-product-officer
  4. Coldewey, D. (2023, July 25). OpenAI scuttles AI-written text detector over ‘low rate of accuracy’. Retrieved July 27, 2023, from https://techcrunch.com/2023/07/25/openai-scuttles-ai-written-text-detector-over-low-rate-of-accuracy 
  5. De Vynck, G. (2023, May 30). ChatGPT ‘hallucinates.’ Some researchers worry it isn’t fixable. The Washington Post. Retrieved June 1, 2023, from https://www.washingtonpost.com/technology/2023/05/30/ai-chatbots-chatgpt-bard-trustworthy 
  6. Edwards, B. (2023, July 14). Why AI detectors think the US Constitution was written by AI. Retrieved July 28, 2023, from https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai/3 
  7. Fowler, G. A. (2023, April 3). We tested a new ChatGPT-detector for teachers. It flagged an innocent student. Retrieved April 4, 2023, from https://www.washingtonpost.com/technology/2023/04/01/chatgpt-cheating-detection-turnitin 
  8. Klee, M. (2023, June 6). She Was Falsely Accused of Cheating With AI — And She Won’t Be the Last. Retrieved June 7, 2023, from https://www.rollingstone.com/culture/culture-features/student-accused-ai-cheating-turnitin-1234747351
  9. McAdoo, T. (2023, April 7). How to cite ChatGPT. APA Style Blog. Retrieved June 12, 2023, from https://apastyle.apa.org/blog/how-to-cite-chatgpt 
  10. Myers, A. (2023, May 15). AI-Detectors Biased Against Non-Native English Writers. Stanford University Institute for Human-Centered Artificial Intelligence. Retrieved July 18, 2023, from https://hai.stanford.edu/news/ai-detectors-biased-against-non-native-english-writers 
  11. Sands, L. (2023, April 6). ChatGPT falsely told voters their mayor was jailed for bribery. He may sue. Retrieved May 30, 2023, from https://www.washingtonpost.com/technology/2023/04/06/chatgpt-australia-mayor-lawsuit-lies 

Leave a Response