🤖 AI Vs. AI Detection Tools: Real-Life Testing

support@bulkcpa.com December 29, 2022 Learn 0

Regardless of your position on AI Content, if you are a web publisher, you are going to be dealing with it in 2023.

It’s too tempting for your freelancers, content services, or even in-house content creators not to test the water with it the next time you give them a boring SEO assignment because the keyword is showing low difficulty in Ahrefs’ keyword explorer.

But if you’re being served AI content, you probably want to know if it is AI content.

So I tested the AI Detection tool Originality.ai in many difficult situations to see if it catches content generated by GPT-3 and ChatGPT.

Can AI help you catch AI?

As you will see, the results were… surprising .

Test #1: Straight Up Copy & Paste AI Content

For the first test, I wanted to give the tool an easy task to gauge its capabilities.

So I went to Jasper.ai and straight up prompted it with:

“write a short article about 5 affiliate marketing mistakes”

Here’s the output I got. It was… generic & bland, to say the least.

Text output by Jasper.ai

01. Forgetting that the core of affiliate marketing is generating leads with customers. Affiliate marketing is not just about thinking in terms of clicks and sales, it’s more than that, as affiliates need to engage their audience and become a credible partner for them, otherwise they will fail at this business model.

02. Relying on one or two affiliates to make the vast majority of their sales. You need to create a network of quality affiliates who will make your business thrive, and not just rely on one or two partners who might end up leaving you at some point in time.

03. Not tracking conversions with affiliate tools such as custom affiliate links, cookies and commissions. It’s crucial that you track your affiliate sales with relevant tools in order to know how much money these affiliates are generating for your business.

04. Failing to promote their affiliate programs on multiple channels and platforms. Affiliate marketing is a strategy that needs to be deployed at different levels, so make sure to include several tactics such as newsletters, blog posts, social media updates and more that can help you reach a broader audience.

05. Not having a clear plan in place when it comes to maintaining their affiliates. If you want your affiliates to be successful then you should create a system where they can report back on their progress, ask questions or get the support they need whenever necessary. A winning formula for affiliate marketing is all about having the right partners who can make your business grow. To learn more about this topic and how to become a successful affiliate, check out this blog post.

And unsurprisingly, the tool caught this right away.

I repeated the experiment several times with similar prompts in different niches to ensure it wasn’t a fluke.

It wasn’t. Every single time, the tool caught it and flagged the content as AI-generated.

It’s safe to say the tool will catch the most basic AI prompts, and I think it may spell bad news for programmatic SEO disciples.

Test #2: Edited GPT-3 Content.

For the next test, I wanted to give Originality.ai something a little more challenging .

I won’t share the content I used because it was taken from one of our sites that ranks quite well, and I’m not exactly interested in outing it today, sorry .

But to give you some context, the content I pasted in was generated by Jasper (GPT-3) and edited by one of our in-house writers to fix its inaccuracies, improve it and make it more engaging.

So it’s part human, part AI, and I wanted to know how the tool would handle that.

And I couldn’t fault it again. The tool caught the more “human” part of the content while understanding it is mostly AI-generated.

Test #3: An Article Submitted By A Freelance Writer For Authority Hacker.

The next trial was the blog post about mattress affiliate programs written by one of our freelancers for Authority Hacker.

In principle, this was fully human-written, but I was curious to see what the tool had to say.

So I took a snippet of it and submitted it to Originality.ai.

And the tool did find the content to be original/human-written.

I must admit I had a small sigh of relief when seeing this.

I suspect the small percentage of AI content may be due to the usage of Grammarly.

Test #4: Emulating Brian Dean’s Writing

It aced all the tests so far, so I decided to make things a little more difficult.

I pasted a Backlinko article in ChatGPT and told it to write an article about another SEO tactic in the same style.

The result was surprisingly good in style, but I’m wondering if it’s not just a rewrite of something Brian Dean already talked about.

Anyway, here’s the text I submitted to Originality.ai:

Text output by ChatGPT

Reverse Outreach: A New Approach to Link Building

Link building is an essential component of search engine optimization (SEO), but many traditional outreach-based strategies do not scale well in competitive niches.

This can make it difficult to outrank competitors and gain the valuable backlinks needed to improve search engine rankings.
One solution to this problem is a new approach called “reverse outreach”.

Instead of reaching out to bloggers and journalists, reverse outreach involves creating content that is optimized for keywords that these individuals are likely to search for when researching or writing an article.

By ranking highly for these keywords, your content is more likely to be discovered and cited as a source by bloggers and journalists, leading to valuable backlinks without the need for extensive outreach efforts.

To implement reverse outreach, follow these steps:

1 – Identify “journalist keywords” – these are keywords that journalists are likely to use when researching or writing an article.

2 – Create high-quality content that is optimized for these keywords and provides valuable information or data that journalists can use in their articles.

3 – Monitor your rankings for these keywords and make adjustments to improve your visibility and attract backlinks.

4 – Track and analyze the backlinks that your content generates to understand what is working and where you can improve.

By following these steps, you can implement a reverse outreach strategy and generate valuable backlinks without the need for extensive outreach efforts. This can help you improve your search engine rankings and gain a competitive edge in your niche.

And it fooled Originality.ai.

Then I wanted to know if the result was due to using ChatGPT instead of Jasper or because of the method.

So I prompted ChatGPT to “write me an article about 5 affiliate marketing tips” and submitted it to Originality.ai.

And it got caught.

Conclusion / TLDR

Most of the time, the tool tells the right story and detects AI content.
However, it can be fooled (see test #4)
You probably only should question writers if several pieces have been flagged by the tool.
Several users on Twitter are already boasting about specific settings you can use to fool the tool.
However, let’s be honest. Most AI-generated content doesn’t have much thought behind it and will probably get caught.
The fact that this is viable at a small scale as a SAAS tool doesn’t mean it’s viable for Google in terms of costs and value, especially considering the fair rate of false positives.
Nonetheless, if you add this to the fact that OpenAI is working on a watermark system for AI content, this can lead us to question the long-term viability of the new “programmatic SEO” trend.

If you want to discuss this topic, join us on the Twitter thread.

🤖 AI Vs. AI Detection Tools: Real-Life Testing