News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.
Pangram costs $20/month for 300,000 words / month, so a small journal receiving 20-30 10,000 word articles per month would…
@ The Pacific: Good questions. My point isn’t that journals can’t adopt that stance, only that I won’t referee for…
It would be a shame if bad philosophy of mind got us all killed. AI can absolutely have desires and…
Does AI really pose an existential risk? Answer: Yes. Reason: There is a pattern of people leaving these companies (foregoing…
This is a load of cobblers. Philosophers need to read more Searle & watch less 2001. Computers, however fancy, are…
I wonder how feasible and time-intensive it would be to make it a standard policy check for undisclosed AI usage.…
One provocative read (if you can get past the sensationalistic headline) is econ blogger Noah Smith’s substack post from last…
A reader calls my attention to this article about Pangram. Curious to hear from readers about their experiences with AI-writing detection programs, whether Pangram, or others.
Pangram’s outputs match my judgment about whether something is AI more reliably than any other tool’s. It is the most widely respected “is this AI?” tool I know of, and my first choice when I want a second opinion (or to evaluate something I haven’t read).
My job is primarily working with AI, so I pay a lot of attention to this area, but I have no special expertise in making or evaluating this specific kind of tool. Others might know more.
I just tried Pangram out as a test. I uploaded two chunks of text that were almost entirely AI-generated using ChatGPT 5.4 (279 words and 506 words) and it rendered a “100% human-written” judgment.
I uploaded another chunk of text that was written with a relatively careless prompt (basically “Write a 500-word essay on the relationship between partisanship and ideology in the United States”). I pasted the result into Pangram and got a “100% AI-generated” judgment. I tried going back to ChatGPT to ask for a revision in a more human register, but the subsequent two checks were also flagged as 100% AI-generated.
One difference might be that the two initial versions were generated based on text I had already written. I loaded text from my papers, asked ChatGPT to rewrite them around a new idea, and pasted the output. So, perhaps the “complete revision” of human text still retains some human signature. It is promising that text generated from scratch gets flagged.
I have no expertise in this, but i have been looking for such a program for student papers. My understanding based on my research (including conversations with experts on this) is that to date there is no reliable software currently available to detect this with sufficient accuracy to use as evidence for honor code violations. FWIW.
I should clarify that this was the conclusion of university counsel as a matter of policy as well. But of course especially in this space, this conclusion needs to be monitored and reevalauted as technologies evolve.
My experience on Pangram is that it does not (seem to) give many false positives, but that it does allow through a good number of false negatives. Given the hoops we at my place of employment have to jump through to deal with these cases, that is probably a good thing, but it’s less than completely satisfying.
There is a piece of flash fiction I fed into Pangram which Chat helped me with some filler, as my imagination is strong at original ideas but poor at the stuff of everyday life. It surprised me Pangram rated my story “And Lincoln will Rise Again” as 100 % mine when Chat agreed it was 90 % mine and 10 % it’s. I did assimilate it deliberately into my voice. That may play a part.
Leave a Reply