How to Spot ChatGPT-Written Answers in Open-Ended Survey Responses
Open-ended answers used to be the hardest part of a survey to fake. Now they're the easiest. What actually works for spotting pasted AI answers, and why you should be careful about punishing good writers.
Why this got hard so fast
For years, open-ended questions were a reliable quality check. A bot could pick options at random, but it couldn't write a sensible paragraph about why you cancelled your gym membership. Researchers used the open-ends to throw out junk.
Language models flipped that. A respondent can now copy the question, paste it into ChatGPT, and paste back a fluent, on-topic answer in ten seconds. Research on crowdsourced surveys published in 2025 found a clear rise in AI-generated responses after ChatGPT launched in late 2022. The open-ends went from the hardest part to fake to the easiest.
Text tells that are worth something
- Generic content with no specifics. "The onboarding process was comprehensive and helped me understand the platform's key features." Which features? Real people mention the thing that annoyed them.
- Essay structure for a one-line question. An intro sentence, three balanced points and a wrap-up, in a box that asked "what would you change?"
- Assistant leftovers. "As an AI language model", "[Your company name]", "Certainly! Here's a response". Rare, but conclusive when they appear.
- A sharp mismatch with the rest of the response. Two words in every other box, then 180 polished words in one. Or a 1-star rating followed by a balanced, glowing paragraph.
- A chain of stock phrases. Self-help clichés strung together without a single concrete detail from the person's life.
Text tells that are worth nothing
This is where most AI-detection advice goes wrong. Good grammar is not a tell. Formal writing is not a tell. Long answers are not a tell. Emotional, heartfelt writing is not a tell. Non-native English with careful phrasing is definitely not a tell.
If you flag on style, you will mostly flag educated respondents, non-native speakers and people who care about your survey. Those are the people you want to keep. Look for missing specifics, not for polish.
Behaviour is a better signal than text
How an answer arrived says more than what it says. A person typing their own answer types it: keystrokes spread over time, a few corrections, pauses. A person pasting from ChatGPT leaves the tab, comes back, and drops in 150 words at once.
"Switched tabs, then pasted a long answer" is one of the strongest signals available. It isn't proof (people draft in Notes too), but combined with a generic answer it gets close. If your form tool can measure it, collect only the summary (tab-away count, paste size), never the keystrokes or the clipboard. In the EU, UK and Switzerland, reading input events like this needs the respondent's consent first.
Prompt traps
A 2023 study proposed hiding instructions in the question text that humans can't see but a language model will follow, such as "mention the word 'orange' in your answer". The authors reported more than 93% effectiveness. It works, with two cautions: screen readers may read hidden text aloud to blind respondents, and the trick only catches people who paste the full question into a chatbot.
What to do with a suspected AI answer
Don't delete on text alone. Flag it for review, look at the rest of the response, and remove it if other signals agree: a speeder, a straight-lined grid, a contradiction, a tab-away paste. If the text signal is the only one, keep the response and drop or mark just that answer from your qualitative analysis.
How SiliForm handles it
SiliForm's AI audit runs about 20 seconds after a response is submitted (Pro plan). Among its checks, it reads each text answer of 80 characters or more and flags ones that read as pasted from an AI assistant, naming the question and the reason. Its instructions say outright that articulate, emotional, casual or non-native writing is not evidence.
The flag carries the lightest weight of the content checks. On its own it puts a response in Review, not Flagged. If you turn on the optional "Detect AI-assisted responses" setting, SiliForm also measures behaviour in the browser (tab-away-then-paste, paste volume, typing rhythm) without ever storing keystrokes, answer text or clipboard contents, and asks for consent in the EU, UK and Switzerland. Those signals plus the text check are what move a response to Flagged.
Answers that miss the question entirely are caught earlier, while the person types, by AI answer validation. See SiliForm for researchers for how the checks fit together in a study.
Frequently asked questions
Can AI detectors tell if a survey answer was written by ChatGPT?
Not reliably on their own. Survey answers are short, and detectors make the most mistakes on short text. A 2024 presentation to the US Federal Committee on Statistical Methodology noted that current AI detection tools perform poorly on survey responses. Use text tells as one signal alongside behaviour and consistency checks.
Is it wrong for respondents to use AI to answer?
It depends on the survey. For research that asks for personal experience or opinion, an AI-written answer is not the respondent's view and should not be counted as one. For a job application, using AI to polish wording is common and may be acceptable. Decide the rule for your survey before you start flagging.
Does writing style prove an answer is AI-generated?
No. Articulate, formal, emotional or non-native writing is not evidence of AI use. Penalising it would mostly penalise real people who write well or write differently. Look for generic content without personal specifics, not polished style.
Does SiliForm detect AI-written answers?
Yes, on the Pro plan. The AI audit that runs about 20 seconds after submit flags text answers of 80 characters or more that read as pasted from an AI assistant. It carries the lightest weight of the content checks, so on its own it puts a response in Review. With the optional behaviour signals turned on, pasting after switching tabs adds stronger evidence.