I Tested Pangram to See If It Can Really Detect AI-Generated Content Like It Claims ...Middle East

News by : (Live Hacker) -

You'll find arguments on both sides of the debate as to whether or not Pangram's claims stand up. After a short story was flagged by Pangram and then judged to be free from AI, the detection engine has been accused of bias against text from minority writers.

As the AI detector has recently added AI image and video detection to its capabilities, it seemed a good time to put Pangram's analysis skills to the test for myself. Here's how it performed in the three different content types.

Can Pangram really detect AI writing?

For the AI content, I rotated between ChatGPT, Gemini, and Claude, and got the LLMs to write 10 articles with the same headlines and the same approximate number of words, for a tech-savvy audience. I even asked the AI apps to vary their language use, so that the output would "read less like an LLM has written it."

Pangram correctly flagged up AI text. Credit: Lifehacker

What's more, Pangram was certain about its results. All of the human writing was marked as highly likely to be human, and the vast majority of the AI writing was marked as highly likely to have come from an LLM. There were only a handful of phrases or sentences across thousands of AI words that got a "medium" rather than a "high" ranking for AI.

Now, there have been some edge cases where Pangram has got it wrong—like a poem repeating the same six words thousands of times—but as far as this sample goes, it can't be faulted. Clearly, my writing is too idiosyncratic and too clunky to be confused for the smooth sheen of AI-produced prose.

Lifehacker articles passed the 100% human test. Credit: Lifehacker

However, I was able to fool Pangram to some extent by alternating AI paragraphs with human-written paragraphs. The text was flagged as AI, but the percentage estimate was typically lower (20-25%) than what it actually was (around 50%). If you're offloading some but not all of your writing to AI, some of it might slip though as real.

Can Pangram really detect AI images and video?

Again, all the human-made content was recognized as such, with a very high level of certainty—whether it was complicated crowd shots or simpler images of natural scenes. Presumably, there are even more signals to go off than there are with writing, including camera movements, colors, style, consistency, and composition.

Pangram was less sure about images mixing AI and real photos. Credit: Lifehacker

Here, too, the AI identification was spot on, with all of the artificial content recognized as such, to a high level of confidence. This is perhaps not surprising: I think AI images and especially AI video still have more "tells" than AI writing, especially if you've seen a lot of it, so I wasn't expecting Pangram to fail here.

I did try experimenting with real photos that had a little bit of AI in them, what with tools like Gemini and ChatGPT now so good at image editing, as well as creating pictures from scratch. These were marked as "unsure" by Pangram, though to cut it some slack, only a small amount of the images were altered. 

Pangram proved adept at spotting the tells in AI videos. Credit: Lifehacker

Increase the level of AI, and the detection gets more reliable—but if you're only using AI for a small part of whatever it is you're making, it's less worthwhile using AI tools anyway. As far as these tests go, Pangram's claims about accuracy check out, making it a useful tool if you need to ensure something is AI-free.

Hence then, the article about i tested pangram to see if it can really detect ai generated content like it claims was published today ( ) and is available on Live Hacker ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.

Read More Details
Finally We wish PressBee provided you with enough information of ( I Tested Pangram to See If It Can Really Detect AI-Generated Content Like It Claims )

Last updated :

Also on site :

Most Viewed News
جديد الاخبار