We Can’t Agree On What Counts as “AI-Written”

AI allegations have taken over the publishing world: there’s a new scandal every week. In all the discourse, though, we seem to have skipped a step—what actually counts as “AI-written”?

AI and LLMs (Large Language Models, like ChatGPT) are being used in every part of the writing process, from brainstorming to research to editing to sentence construction. So, where do we draw the line? And which AI-assisted works are we willing to read?

I put together a survey with nine questions to get a gut check, asking respondents to select for each question whether the author’s use of AI/LLMs would make the finished work “AI-written” and whether they’d be open to reading it.

This is not a scientific survey; most respondents are Book Riot Newsletter subscribers, who are not a representative sample of the general public. Instead, it’s a look into how dedicated book lovers are perceiving AI-assisted work at this moment in time.

Used an AI-powered spellcheck/grammar tool like Grammarly.

I structured the survey to go through a spectrum of AI-assistance, ending with the most straightforwardly AI-written work. As expected, most of us don’t think using an AI-powered spellcheck/grammar tool equates to a work being AI-written—which is good, because full disclosure: I used Grammarly to edit this article. (And ignored its suggestion to change that em-dash to a comma.)

Still, 11% of respondents disagree and categorized this hypothetical work as AI-written.

Used AI/LLMs for research while writing a nonfiction essay/book.

One of the most controversial corners of the AI writing debate right now is about using AI/LLMs for research. AI-assisted research can take a lot of different forms, so this is likely worth drilling down on. About three-quarters of respondents said using AI/LLMs for research doesn’t make the work AI-written, but regardless, the majority of respondents wouldn’t be interested in reading it.

Ran their writing through an LLM for advice on structure.

This was the most surprising result of the survey for me. We’re split almost exactly 50/50 on whether asking an LLM for advice on structure means the finished work is AI-written. I’ll be honest—I did not expect so many people to object to this.

The results suggest that readers are very skeptical not just of LLMs forming the sentences of a work, but also of LLMs being used as an editor.

Asked an LLM for a story idea, but wrote it themselves.

On the other hand, there seems to be more acceptance of LLM use in the brainstorming phase: if a story idea comes from an LLM but is written by a human, most respondents wouldn’t consider it AI-written, and most would be open to reading it.

Initially, this surprised me, but it does reflect one of my pre-existing beliefs: ideas are cheap. Generating ideas is easy; it’s the execution of those ideas that matters.

Asked an LLM for improvements on a story they wrote themselves and incorporated the suggestions.

Two-thirds of respondents said that getting advice on improvements from an LLM and incorporating those suggestions means the finished work is AI-written. This was a larger majority than I expected, which again reflects the unease readers seem to have with LLMs as editors.

Wrote a skeleton draft (bullet points) of a story and had an LLM expand it into the finished story.

Keeping with the “ideas are cheap” theme, the vast majority of respondents think that creating a skeleton draft of a story and having an LLM write the full version means the finished work is AI-written. On the other hand…

Had an LLM write a first draft of a story and then completely rewrote it in their own words.

Almost 90% of respondents think rewriting an LLM’s story in your own words still makes the finished product AI-written, but it does seem to be perceived as slightly more human than the other way around.

Had an LLM write a first draft of a story and then rewrote approximately half of the story in their own words.

As I expected, the last few scenarios are fairly straightforwardly categorized as AI-written. If you take an LLM’s story and completely rewrite it, as in the previous question, 87.5% categorize it as AI-written. If you only rewrite half the story, 98% of respondents say it’s AI-written.

Had an LLM generate the entire story and did not edit or change anything about it afterwards.

I had to include a calibrating question, and luckily, we’re on the same page here: 99.1% of respondents said this would be AI-written. (3.1% said they’d still be open to reading it.) Presumably, that last 0.9% is a little lost.

---

Here are my takeaways from the survey results:

We are not on the same page: even within Book Riot circles, we have very different ideas about where to draw the line in what counts as “AI-written.” Ideas are cheap: readers are less interested in where the idea came from than in who wrote the words and executed the idea. We need to talk about LLMs as editors: most of the discussion of AI-assisted work has been focused on writing, but readers are extremely skeptical of LLMs being used in the editing phase as well. We need to discuss what use of AI (if any) is acceptable at this part of the writing stage, too.

For more like this, sign up for the Book Riot Newsletter!