Your Ultimate Guide To Writing Amazing Survey & Poll Questions

Whether it is your first time creating a survey on your own, or you are a seasoned research professional, one thing is for sure: if you can’t write good survey questions, you won’t get helpful consumer insights.

But just what is a “good” survey question anyway? In this guide, we break down the four key elements that you need to master in order to write better survey questions.

A “good” survey question is:

1. One that achieves your survey and data goals.
2. One that is clear and direct.
3. One that carefully considers user experience.
4. One that is free from bias.

Throughout this guide, we will show you the tactics you can use to make sure you are delivering quality survey questions. To help you along, we will also show you what not to do, so you can easily spot the difference between a good question and one that needs some work.

For example, let’s say we want to avoid bias. Which of these is the better choice?

Set (and Achieve) Your Survey Goals

Everyone wants to write good survey questions–they want to be clear, direct, engaging and get respondents excited about answering. 

But these are not the most important reasons to take care when writing survey questions. 

The reason is if you build a survey with the wrong questions or question types, you could get stuck with a bunch of data you can’t use. 

That’s why it is important to set data goals in advance and tailor your survey to achieve those goals from the very beginning. 

How To Set Survey Data Goals

On the surface, this may seem simple: just state what you want to learn from your respondents and go from there. 

But different surveys deliver different types of data. So you need to know how you plan to use the data you get from your questions first. 

For example, let’s say your goal with your survey data is to create a map so you can continuously survey consumers every month and map their responses over time. 

For this to work, you will need to combine demographic data with raw numbers of respondents and their answers.  

You also want to ensure that your questions are simple and not prone to shifts in understanding based on geographic location or time passing. 

Knowing these data goals, which of these two questions would be better for your survey?

Question Writing 101: Clear, Direct, Well-Tested

When writing anything, clear, direct communication should be the ultimate goal. You want to ensure that, regardless of any flowery language or interactive content features, the core of what you are trying to say is communicated loud and clear. 

With survey questions, this becomes even more important.

Once you have selected your survey goals and you have a good idea of what kind of data you want to collect, the goal of your survey questionnaire is to get respondents to completely fill out your survey as accurately as possible. 

Market research best practices show that it is best to let respondents decide if they want to finish your survey or not. With Pollfish, respondents can escape the survey at any time. Therefore, all responses are voluntary and of higher quality than if respondents were forced to finish the survey. 

What that means for you as the researcher is you have to ensure that your questions don’t drive respondents away. 

Don’t worry, Pollfish is here to help you with that. First of all, each questionnaire is reviewed by our research experts prior to launch. But you can help by following these three steps to survey question writing: be clear, be direct, and test on real people. 

Be Clear

In order to get the types of answers you expect, you must ensure that your target audience will fully understand what you want from them. 

What does that mean? Below are a few common mistakes researchers make when writing their first questionnaire. 

Too Much Jargon

When writing for a niche audience, industry jargon can be a welcome addition. Things like common abbreviations and loaded terms help build rapport and allow you to establish a personal connection with your readers. 

Not so in market research. 

Imagine you are reading an article and you come to some jargon you don’t recognize. All it takes is a quick Google search and you have learned some new terminology.

But if you come to a survey question that contains an acronym or abbreviation you don’t recognize, not only will you feel like you can’t answer that question, you will feel like maybe you aren’t qualified to answer the rest of the survey. You may attempt to guess at the meaning, or escape the survey all together. 

Even if your audience is screened to only be in your industry, don’t assume everyone uses the same terminology you do. Take care when using any kind of jargon, acronym or abbreviation in your survey questions. 

Extra Explanation

Most survey question types are fairly self explanatory. If there are 5 answer choices and the instruction is “Select One”, there isn’t much room for interpretation there. 

But when questions get more complicated, extra explanation is essential. 

For complicated concepts or ideas that respondents may only have some ideas about, a “description” question type can be helpful in clarifying which part of the idea a respondent should focus on.

Matrix questions are one question type where you have to be careful of your labeling and explanation. Let’s take a look at two Matrix questions and see which is the most clear. 

Questions that contain media like images, audio files and videos also often require further explanation. Let’s take a look at the video questions below and see which is the most clear.

If it is necessary to consume all media to understand the question being asked, make sure your instructions clearly state this. You can even include your media as a separate question. 

Create non-leading questions

It can be tempting to write questions in a way that seems obvious to you, but survey questions should not follow the same patterns we have when speaking to each other. 

Although plain language is advised, inserting your own opinions—even subtly— into the survey question can lead respondents to feeling more inclined to agree with the answer you want. 

The second question remains neutral towards either show. Since the goal of your research is to uncover the most accurate data, you want to avoid pushing respondents towards one answer over another.

Make Answer Choices Clear & Distinct

Once you have made sure your questions and instructions are clear and direct, make sure your answer choices are distinct from one another. 

One common reason respondents bounce from a survey is they cannot decide between two similar-seeming answer choices. 

This can also reduce data quality because it may result in respondents offering answers they don’t really mean. 

Although “yes” and “no” can seem like two distinct answer choices, these also don’t offer gradients within the answer options to appeal to respondents who don’t feel strongly or are unsure of an answer. 

We recommend avoiding “yes/no” questions in favor of a Likert Scale or by writing answers out entirely to ensure less ambiguity and reduce bias. 

Which of these questions seems correct?

Be Direct

Good survey questions contain two parts: the question and the instructions. 

For many Pollfish survey question types, some level of instruction is built in. But you want to be sure that when you deliver instructions, you are speaking directly, in clear, declarative statements. 

Using too much passive language and indirect phrasing can create unnecessary confusion. Let’s take a look at the two examples below. Select which you think is the most direct. 

Always keep your Survey Goals in mind

As we learned in part 1, survey goals are at the core of all research projects. To make sure you’re writing the right survey questions to meet the right goals, consider the flow and structure of the survey and how the respondent will be presented with each question. Your goals may also dictate the type of question respondents should be given. 

Example: Survey goal is brand awareness

It’s best to begin a brand awareness survey with an open-ended question for respondents to recall brands that are top-of-mind, prior to being presented with a series of questions that might trigger their recall. 

If the survey goal is press for a specific cereal brand, ranking questions can help uncover consumer preferences.

Ranking questions ask respondents to order answer choices, forcing them to choose one over another. These are best for establishing stronger opinions or validating a strong preference.

The data from a ranking question could be “85% of consumers believe that Trix is the best children’s cereal”, providing a good headline for a research story.

Matrix questions allow respondents to apply a ranking scale towards each cereal, but don’t force them to choose one over another or compare them. These are a better question choice to measure other sentiments.

The possible data to come from a matrix question could be that nearly ¾ of 90’s kids recall Frosted Flakes’ “Tony the Tiger”, however, the Cheerios bee has largely been forgotten.

Test On Real People

While it is important to align with all stakeholders internally on survey goals, you also want to test out your final questionnaire on some real people. 

This can mean either sharing it with co-workers who are not familiar with the research project, or emailing your questions to a friend. 

Getting outside help can be essential in identifying unclear directions that cause confusion for your respondents. 

Respondent Experience: Think Like An App User

Because nearly everyone has an internet-enabled device with them at all times, survey companies have identified the opportunity to use this technology to reach respondents.

Unfortunately, not all surveys are created equal when it comes to offering a good respondent experience on mobile devices. 

Poor respondent experience can have disastrous consequences, impacting your survey response and completion rates, and even impacting the data quality of your study. 

So how can you follow best practices for mobile surveys? 

Build on a Mobile-First platform

By using Pollfish, you are already off to a great start. That is because every Pollfish survey question type is optimized for mobile out of the box. 

Our surveys are not only optimized to fit your mobile screen, they are delivered exclusively inside mobile apps, meaning each question type is designed by app developers to mirror the experience of an app. 

Pollfish also provides a mobile previewer for each question in your survey, so you can test how your question will look on a phone in real time, right inside the questionnaire builder. 

<Maybe add a screenshot of the questionnaire builder here?>

Shorten your questionnaire

Because so many users prefer to take surveys on mobile devices, it is important to think like a mobile user. 

Think about your app usage–do you stay in one place on your phone for a long time, or do you jump quickly from place to place? Do you often have your phone open for hours or do you use your phone in quick bursts? 

Mobile phone users want to move fast. They could be on the go. They could only have a few minutes to kill while waiting in line at the bank. 

That’s why it is important, when you can, to keep surveys short. 

Shorter surveys get faster results and ensure full engagement with your questions from start to finish. 

Our survey experts review every survey before it is published as well, so they may recommend you shorten parts of your study if they feel respondent experience may be impacted. 

Keep a mobile mindset

While all our survey questions will look great on mobile, there are a few things you can do to make your questionnaire more engaging to the mobile user. 

Start by favoring question types that are easier to answer on mobile phones. Multiple-choice questions that users can tap, rather than open-ended questions where typing is required makes answering easier for users. Question types like rating stars and using video can keep users engaged and motivated throughout your survey. 

Keep in mind that while many survey tools that offer a “mobile survey” experience also offer matrix questions, you’ll want to verify that these have been designed for mobile distribution. 

Matrix questions in a typical survey present as a table where respondents are asked to rank a series of answers. In mobile, these don’t translate into a good experience for respondents. 

Ensure that your mobile matrix question utilizes vertical scrolling and doesn’t require respondents to zoom in or drag the table horizontally to see all the answer possibilities.

While more qualitative questions may be helpful to provide more context and emotion to survey respondents, mobile respondents would rather tap or swipe than type. 

If you were a mobile user, which of these would you prefer to answer?

Prevent Survey Bias

One of the most important parts of building the perfect survey is reducing survey bias. 

From Your Ultimate Guide To Survey Bias

Put simply, survey bias impacts data quality. By eradicating survey bias (as best you can), you build trust in the data you’ve collected. 

That’s no small thing, as companies across the world are using consumer insights data to inform everything from content creation to product timelines to strategic planning.

So how do you remove bias from your questionnaire? 

Well, for starters, it is important to remember that preventing all survey bias is impossible. 

That’s right. We are all biased. We have unconscious thoughts that cause us to subtly indicate our true feelings to those around us. And that applys to writing survey questions as well. 

The key is to do everything you can to remove the most pernicious forms of bias from our surveys so we can ensure top-quality data. 

With this section, I’ll take you through a few of the more common ways to avoid bias in your questionnaire. 

Remove Double-barreled questions

Double-barrelled questions happen when researchers blend two questions into one, and then allow for only one answer. This creates a fallacy for the respondent and often results in incomplete or biased answers. 

Can you tell which question is biased here? 

Avoid Assumptions

Depending on how you write a question, you could be assuming things of the respondent, or even forcing them to give an answer they don’t believe. 

One way researchers fall into this trap is by asking loaded questions. Loaded questions make multiple assumptions about a person and then force them to answer a question based on those assumptions. 

Can you spot the Loaded question? 

Don’t Use Biased Language

Similar to Loaded questions, biased language (sometimes called Leading questions) pushes respondents towards one response or the other. 

This removes the impartial nature of a survey and influences respondents, leading to poor data quality. 

Can you spot the Leading question?

Do Your Best To Include All Available Choices

As a general rule, you want to avoid forcing respondents to give an answer they don’t really mean. 

So when you are writing a closed-ended survey question with a finite answer pool, make sure you think hard about additional answer possibilities that may not come immediately to mind. 

Once you have done that, there are two additional ways you can reduce bias on these kinds of questions. 

The first is to add an open-ended “Other” field, where respondents can include any additional choice you may have forgotten. If there is enough consistency in responses, you may even be able to manually quantify the data. If not, you have still provided some much-needed context to the current answer choices. 

If you want to stick with your closed-ended question type, you can make it clear in your instructions that respondents are to select the answer closest to what they think. This is imperfect and provides a tough-to-quantify variable to your data. But it also will keep respondent experience and completion rates up.