How repetitive questions can ruin survey results

Want accurate survey results? Stop asking repetitive questions.

In order to glean as much information as possible, many surveys are stuffed with questions that vary only slightly from one another in terms of structure. But according to a recent study led by Ye Li of UC Riverside and cited in ScienceDaily, repetition can make survey respondents less alert.

If you think about it, this makes sense. Imagine being asked something similar over and over - for instance, a small child keeps bombarding you with “why” questions. It’s very likely that you’ll eventually just go on autopilot, giving absent-minded answers and hardly listening to the questions.

And yet, the article explains, surveys in many fields often contain lists of questions whose structure hardly varies. For instance:

- Do you take medication on a daily basis?

- Do you take multiple medications on a daily basis?

- Do you take medication that causes you to suffer side effects?

And so on.

As they continue to respond, the study found, respondents become less and less alert and reactive to these similar questions.

According to the scientists who conducted the study, it comes down to human nature. We eventually adapt to new situations, and so in this case, our brain tells us that we might as well respond to a certain type of question in the same way.

These findings bring up a paradox in survey writing: as Li and his team put it, “The more you ask, the less you get.” In other words, asking fewer questions will actually give you more valuable information, because responses will be more accurate.

While asking similarly structured questions may seem like an easier, more streamlined way to write a survey, it’s fortunately far from necessary. For survey writers who might feel stumped, the Science Daily article suggests additional techniques that can improve survey results. These include:

- Changing things up. Survey writers should consider changing the formats of tasks and questions.

- Using padding. Add a few filler questions to break the rhythm of a series of questions that might be too repetitive.

- Examining trends in results. This could be done via a technique called process tracing, which analyzes both the quality and quantity of responses.

If you’re not a survey writer, you may wonder why it’s taken so long for this kind of conclusion to come to light. Maybe it’s a sign of how professionals can be so focused on the goal that they lose sight of important elements along the way. It’s a phenomenon seen again and again in the healthcare and pharma fields. For instance, we see it in doctors who are so lost in paperwork and trying to fit patients in that they lose their bedside manner, or in patient education materials that may be translated but truly need to be transcreated in order to be fully understood by the local population.

At least in the case of surveys, there are relatively simple solutions. Hopefully, by improving survey questions, results will also improve, making things a lot better for both researchers and those who could benefit from their findings.

Image source

Contact Our Writer – Alysa Salzberg