Straight-lining and Satisficing in Surveys: What They Are and How to Catch Them

Satisficing: the idea behind both

Answering a survey question properly takes four steps: understand the question, recall what you think, form a judgment, and map it onto the options. The survey researcher Jon Krosnick called the shortcut version "satisficing": skipping some of those steps and picking something that will do.

Satisficing is not lying. Most satisficers aren't trying to fool you. They are tired, or bored, or the survey is long and the incentive is fixed. The result is the same, though: answers that reflect the effort of the respondent more than the opinion of the respondent.

Straight-lining: the easiest form to see

Show someone a grid of ten statements with a five-point scale, and the cheapest path to the end is a vertical line of clicks down one column. That is straight-lining, and it shows up in the data as a row with the same value repeated.

A simple rule catches most of it: four or more grid or rating answers in the same option position. Count each row of a matrix as one answer. If a matrix has six rows and all six are in one column, that is a straight-line.

Variants are worth knowing: zig-zags (1, 2, 3, 4, 5, 4, 3...) and "Christmas tree" diagonals. Both are rarer and harder to rule on. If you see them, look at the rest of the response.

Agreeing with everything: the form that hides

A more careful satisficer varies the position but always picks the friendliest answer. Strongly agree, Very satisfied, Extremely likely. Their row doesn't look straight, so a straight-lining check misses it.

The check here is "how often is this the most positive option?" If someone picks the most positive answer on 90% of six or more rating questions, that is worth a look. It works best when your survey includes some negatively worded items, because a person who strongly agrees with both "support was fast" and "support was slow" is not reading.

When it is honest

A 2020 paper in Survey Research Methods by Reuning and Plutzer made an uncomfortable point: better questions can produce more straight-lining, because well-designed items that measure the same thing will, correctly, get the same answer. Throwing out every straight-liner throws out some of your most consistent respondents.

A few practical rules follow from this:

• Don't remove a response for straight-lining alone. Look for speed, failed attention checks or filler text alongside it.
• Pay more attention to straight-lining across items that should disagree (reversed wording) than across items that should agree.
• Check the share of straight-liners per grid. If one grid has far more than the others, the grid is probably the problem.

Design that invites less of it

Prevention is cheaper than cleaning. Big grids are the main cause of straight-lining, so break them up. Mix in reversed items. Keep surveys short enough that people don't start skimming in the second half. Showing one question at a time also helps, because there is no column to click down.

How SiliForm checks for both

On the Pro plan, SiliForm checks every response at submit for straight-lining (the same option position on four or more choice questions, with each matrix row counting as one, or four or more rows of a single matrix in one column) and, when that doesn't fire, for "always agreeable" answers (the most positive option on at least 90% of six or more eligible questions).

Neither check removes a response. Each lowers the 0 to 100 quality score and shows its reason on the response, so a straight-liner with otherwise thoughtful answers stays in Review while a straight-liner who also sped through lands in Flagged. And because SiliForm forms are conversational, showing one question at a time, there's less of a grid to click down in the first place. See SiliForm for researchers for the rest of the checks.

Frequently asked questions