AI content detectors don't work. Here's the alternative.

AI content detectors have become part of the content approval workflow in many companies. While there are no statistics to tell us how many companies use them, discussions about AI detectors like Copyleaks, GPTZero, DetectGPT, SurferSEO, Originality.ai, and Turnitin are pretty common on Reddit.

The increase in what's known as AI slop is driving companies to use these tools to screen content they've paid for. But in all cases, the tools are unreliable.

Universities and schools were early adopters in the late 2010s. There's a legitimate reason to want to vet students' work, and plagiarism detection isn't new.

But over time, AI detectors have bled through into business usage.

In parallel, the newest LLMs have rendered these tools useless.

“Current detectors of AI aren’t reliable in practical scenarios,” said Soheil Feizi, an assistant professor of computer science at UMD. “There are a lot of shortcomings that limit how effective they are at detecting. For example, we can use a paraphraser and the accuracy of even the best detector we have drops from 100% to the randomness of a coin flip. If we simply paraphrase something that was generated by an LLM, we can often outwit a range of detecting techniques.” (University of Maryland)

In separate studies, AI detection tools have flagged sections of the Bible, the Qur’an, the US Constitution, and Shakespeare’s Macbeth as being partly or entirely written by AI. Turnitin’s AI detector also flagged an international student’s paper as being AI-generated.

Sadly, this is a pattern often seen with AI detectors; non-native English speakers bear the brunt of their mistakes. Writers who are less proficient tend to use less varied language. (The student in this case was able to prove the detector was wrong, but others may not be so lucky.)

In recognition of these ongoing failings, OpenAI removed its AI detector from its website in 2023. From its FAQ: "Do AI detectors work?" "No, not in our experience."

Using inaccurate tools causes harm to freelancers, students, and businesses. A cheap online detector or Chrome extension is enough to kill their earning potential and cheat them out of a qualification they worked hard for.

So what's the solution? Ultimately, if you're trying to detect the use of generative AI, you have another goal in mind. Knowing your goal will reveal a better way to solve the problem.

Consider your reasons for detecting AI content

If you’re reliant on an AI detector, the first thing to understand is why you’re using one.

From this point, I’ll set aside academic concerns and focus solely on business usage.

Your reasons may fall into one of these categories:

1. You're paying a writer to write content, not generate it!

This is an entirely reasonable point. If you wanted AI-generated content, you could do it yourself. Paying someone else to generate it doesn't make sense.

However, if you pay a writer a decent rate, and you know that writer has a track record of writing good content, you don't need a detector. Good writers produce content because they get a kick out of doing it. They enjoy educating and communicating. Most don't want to outsource their talents to an LLM.

In my experience, paying a really good writer to produce an article is cheaper than paying them to get the same results solely from AI. Prompting, re-prompting, and stitching all the sections together takes longer than just writing it from scratch.

Of course, if a writer really needs to slash the time it takes to write articles, you might want to look at whether you’re paying them enough.

2. You’re worried about ranking issues

This is also perfectly legitimate. If you don't live and breathe SEO, you could be forgiven for thinking that AI content has ruined Google's results.

But in my experience, Google does not give a fig about whether something is AI generated or not. Google’s goal is to sell ads and train its AI models. It will rank literally anything to do that, including school assignments in Google docs:

More to the point, Google has no possible way to detect AI content, any more than OpenAI can. It simply has no way to tell. So even if it wanted to wage a war against AI slop, it would have no mechanism to do that.

What matters to Google in 2024? Engagement signals. Clicks, dwell time, scroll depth. Those metrics are your new baseline.

3. You’re worried about turning off your audience

I'm guilty of being a snob about generated content. When I see telltale words like 'delve' and 'digital landscape', I can’t click the back button fast enough.

But readers aren't using AI detectors to judge your content. They judge quality by reading and deciding for themselves. And you can too. As an expert in your niche, you are the best judge of quality.

When you receive content, you should be looking for:

• Original insight, opinions, and evidence of personal experience - this is our good friend EEAT, and it's as relevant as it ever was
• A consistent ‘voice’ and tone that sounds like the writer you hired
• A level of insight that would make this content worth sharing with a friend
• An absence of words and phrases that sound unnatural or are repeated
• A consistent narrative thread.

If you outsource this value judgement to an AI detector, you will risk losing the writers you depend on and your readers at the same time.

Replace AI detection with human communication

The best solution to this problem is an honest and professional conversation about what you want and expect.

Writers should be using AI to save time on mundane tasks. That's the norm within most businesses, so it should be expected in freelance or contract relationships too. As long as it doesn't compromise privacy or IP, efficiency is beneficial. We can't hold back time by pretending AI doesn't exist.

We can move forward by understanding how AI can be used to everyone's benefit. Here are some questions to ask content creators when hiring:

• Ask them upfront what they use AI for. There are plenty of good reasons to use AI; I'll run this through Claude to check for typos before I publish it, for example. Get on the same page about the tools your writer uses and understand why they work that way. (Since AI tools can be expensive, you should not expect the content creator to lower their fees if they use AI for workflow improvements.)
• Don't push writers to use tools like Grammarly. Automated content checker can sometimes prompt writers to include edits that look unnatural. In my experience, the free version of Grammarly is helpful, but the Pro version makes some odd suggestions. While we want to strive for good grammar, a little imperfection is perfectly fine, and a slightly quirky tone is what makes content human.
• Be willing to pay the market rate for human-written content from a writer who has an established track record. This will also help to position your brand as an authority.
• If you don't want to allow AI to be used, add that to your terms. Some clients have legitimate confidentiality concerns that are perfectly valid. A good agency or freelancer will always take those requirements into account.
• If you have doubts about whether the content you're paying for is human-written, simply ask for the next assignment to be created in a Google Doc so you can see the revision history for yourself.

There is a sweet spot where AI enhances creativity but doesn't replace it. That's where we need to be. Whether the creation process involves AI tools or not, you don't need a detector to tell you whether something resonates with you as a human, and that's the ultimate test.

This content was human-written and reviewed by Claude.