AI Detectors promise to be able to detect content generated by AI. We look at what they are, how they work and possible ways to avoid detection.
What is a ChatGPT Detector?
A ChatGPT detector is a tool that allows you to copy and paste text into it, and the software provides a score of whether the text was written by a human or written by an AI. These products are becoming increasingly popular as educators and businesses attempt to stay ahead of AI generated content.
Can ChatGPT be Detected?
Yes, in many cases ChatGPT can be detected. It’s important to point out that there can be exceptions and many ChatGPT detectors will only provide you with a confidence score of how sure they are that the content was created by an AI chat bot.
The detectors we tested on ChatGPT 3.5, the free version of ChatGPT available from OpenAI is easier to detect than later versions such as ChatGPT-4. There were very significant improvements to the language model between the two versions. ChatGPT-4 access currently requires a paid subscription but the model can be accessed for free by using Bing Chat which will generate long form content from within the Microsoft Edge browser.
We found that the longer the ChatGPT generated content is, the easier it is to detect. Long papers and multiple paragraphs are often enough of a sample size for detectors to work. They have a harder time determining if a sentence or less structured content like marketing tag lines are created by an AI.
AI Detectors vs. Plagiarism Checkers
Plagiarism detectors have been around for years. They are built into a lot of academic tools to make it easier for teachers to upload papers and have them analyzed to determine if the content of the paper was copied from another source.
They are similar to AI Detectors in that they are looking for specific patterns, but AI detectors are much more sophisticated. A plagiarism detector might look for specific paragraphs or sentences that have been used before while AI detectors use complex algorithms to detect the pattern of text instead of the specific sentences or paragraphs.
How do ChatGPT Detectors Work?
Large Language models like ChatGPT from OpenAI are trained on large sets of sample data. This is typically text from openly available datasets, or the internet as a whole. ChatGPT looks at patterns in the text to use as examples of how to answer questions. Because it was trained on text written by humans, the AI model attempts to recreate sentences, paragraphs and pages in a similar way that humans would have written it.
However, services like ChatGPT can be formulaic and have also been trained to use specific types of punctuation, grammar and sentence structures. Tools that have been developed to detect AI written content use text examples from software like ChatGPT and try to learn the patterns of its output.
Like ChatGPT itself, AI detectors rely on training data, either they are separately trained on text created by ChatGPT or they pass data back into ChatGPT or another Language Model.. These services can make use of an API to take the text that a person enters and send it to an AI chabot like Azure OpenAI ChatGPT.
Using ChatGPT to Detect Google Bard
As a fun experiment, we asked Google Bard, Google’s version of ChatGPT to write a paragraph about Harry Truman after his presidency. We then copied the text from Google Bard into ChatGPT and asked if it was written by AI. Incredibly, ChatGPT was able to determine that the text was generated by AI even though it got the initial source wrong.
ChatGPT Detecting Text Written by Google Bard
Why do People Want to Detect AI Content?
In an school setting, teachers want students to do their own work. It’s part of the process of learning and writing papers is an age old, hated tradition. There is nothing fun about having to research and write 24 pages on a topic within a week in a college setting, but it helps students learn how to do research, put thoughts together and become better writers.
Businesses may want to detect AI content because they are paying a freelancer or agency to create original content that has been well thought out and is creative or unique. It’s highly plausible that contractors or employees could take shortcuts and use tools like ChatGPT to create what they should be creating on their own.
Can Google detect ChatGPT content?
With as much money and effort that Google is putting into building out their Google Bard competitor to ChatGPT it is reasonable that they have the capability to detect ChatGPT content along with content generated by other AI chatbots. However, Google has publicly stated that they are okay with AI generated content as long as it provides value and meets the same standards as human generated content for being ranked in search results.
We do not recommend relying on ChatGPT for generating full blog posts with the expectation that they will rank highly on Google. There are many reasons, but the biggest one is that the content will not be as original as someone who is putting a lot of thought and effort into creating truly unique content based on their experiences. That’s not to say that there isn’t a place for AI in content creation though.
Its expected that the search engines like Google will adjust their algorithms and search result layouts. Their very small yet biggest competitor Microsoft Bing has integrated ChatGPT into their search experience. Google will likely do the same with Google Bard. It’s impossible to know if this will change Google’s attitude towards AI generated content in the future.
How do you make AI text not detectable?
If you want to ensure that AI text is not detectable, you will likely have to re-write it yourself. Every person has their own style of writing which will differ from how an AI model re-creates text. Grammatical errors, spelling errors, differentiation in spacing between words can all be clues that text was not written by a computer.
Another approach is to prompt ChatGPT to write a paper in a specific style or at a specific reading level. ChatGPT is capable of explaining complex topics and adjusting the output to a users specific reading level. We have used this technique in the past when researching machine learning prediction models, but it can also be used when using it to write papers.
Can Teachers Detect AI Writing?
Some savvy teachers and school districts will have subscriptions to AI detectors that they use. They typically already check for plagiarism through a suite of education tools that will likely be adding AI detection capabilities in the near future.
Beyond AI tools, it will be pretty obvious to an experienced teacher when a student consistently publishes papers above the average writing level for their age. They will also notice large jumps in writing capabilities and always have the option to have students write papers in class. When no AI tools are available, it will become immediately obvious who has and has not done their homework.
What is the Future of AI Detection?
We are only at the beginning of learning how best to use AI text and content generation tools. These tools will improve over time and produce results that sound more human. The more human that AI generated text becomes, the harder it will be to detect.
Our expectation is that as services like ChatGPT improve, then the detectors will also improve. There could be some laws and regulations placed around declaring if content is AI generated in the future. The entire industry could follow the path of radar detectors. Radar detectors became popular in the 1990s as a way to tell when a police office was up ahead and warned you to slow down to avoid a speeding ticket. Law enforcement responded with better radar guns that were less detectable and the industry has evolved back and forth ever since.
Conclusion
Detecting the use of AI to generate content will become an industry of its own. It will grow and expand along with the use of Large Language Models like ChatGPT and Google Bard. While its possible for students to use the services to cheat, it will become painfully obvious of people that took too many shortcuts and are unable to string words and sentences together in a coherent way.
Students of tomorrow will have to navigate the temptation to cheat, but also the requirement to learn how to best leverage these new tools. Generative AI is not going away and working alongside AI will be a skillset that is required to excel in the workplace.