Why respondent experience matters in mobile surveys

If you’re running a survey, you want to get the best data possible from your respondents. Survey tools will discuss their methodologies, audience sizes, and features, but what many fail to mention is the respondent experience and how your data may be affected by those who are participating in the surveys themselves. 

Mobile surveys are the future   

Nearly everyone has a personal cell phone, tablet or other device type that they will likely have with them where they can be reached, and survey companies quickly identified the opportunity to use this technology to reach respondents.

Mobile offers a more representative distribution of surveys by granting access to qualified respondents outside of a panel. Instead of limiting the survey audience to professional survey takers for incentives, non-panel survey companies were able to leverage the mobile distribution model to expand the representative sample population to nearly the entire planet. 

The mobile delivery model also means that instead of dedicating time to complete a survey or series of surveys at home, a respondent can be out shopping, reading, or waiting in line—completing a survey at a time that is convenient to them and more representative of the environment in which they live their lives. 

These consumers are more engaged in what they are doing and more actively participating in a survey than those who view it as a job or task, something to “get around to” when they have set aside the time. 

Benefits of mobile surveys are clear: the larger the audience, the more selective those who are creating surveys can be about who they want to target (without losing sample size). Consumers completing surveys while actively engaged in them are providing better responses than those who are attempting to quickly get it over with. 

User-experience in mobile surveys

User experience, or UX, is the growing discipline of putting users at the center of interaction design. At its core, UX is about developing empathy for the end-users of a product and creating a more usable, seamless experience for them.

Businesses have quickly recognized the benefits of investing in UX as they see increases in customer loyalty, retention, and positivity surrounding their brand. As it turns out, people are happy to continually use and engage with products that work. 

Most survey tools jumped on the opportunity to reach respondents using mobile but did not consider how consumer behavior would change when presented with a new interface and largely have not embraced the need for a user-centric mobile experience for respondents. 

What this means is that no matter how well-constructed or distributed the survey is, the answers will be compromised if respondents can’t figure out how to complete the survey or leave mid-way because of frustration.

Issues of non-mobile optimized surveys

Poor data quality

As you can imagine, respondents trying to open a survey on their device that has not been mobile-optimized are not giving their best. Their attention is dedicated to zooming in, out, or redirected to a third-party application that takes them away from what they were doing. If they manage to complete it, it’s likely because there is an incentive encouraging them to push through. However, their desire to end a horrible mobile interaction is likely outweighing any thought that goes into their responses, meaning poor data quality for you. 

Low completion rates

Most people are happy to answer a few quick questions, especially if there’s an incentive for them to do it. Survey companies largely use cash, rewards, points, perks—even lightly veiled threats, such as a paywall—to incentivize respondents. But even the best incentives have their limits, and respondents will reach the end of their patience on mobile surveys if they are too long. Mobile surveys need to be quick and to the point to prevent boredom or distraction that can result in low survey response rates.

Questionnaire confusion

Mobile and desktop interactions simply do not format the same way. Unsurprisingly, this also applies to questionnaire presentation. While some question types can easily be translated to a compact interface, others require re-working for mobile optimization in order to reduce respondent confusion.

One example is a matrix-style question type. Presented as a table on desktop, this question type can be a mess on mobile. When horizontal alignment exceeds the limits of the screen, respondents may not see all answer choices or may be confused or frustrated with how to choose the best one for them. 

Choosing a mobile-friendly survey tool

Surveys that don’t make respondents think about how to take the survey result in better answers because respondents are able to focus on the content of the questions. Fortunately, there are survey tools that recognize that respondent experience is directly correlated to data quality.

There are a variety of features that a survey tool with a good mobile respondent experience should offer.

Designed for mobile interface

Survey tools should be able to tell you if they have designed their surveys for a mobile experience. The designers will have considered that respondents are completing surveys by tapping or scrolling with a finger rather than a mouse, working within a smaller screen and that the respondents are likely on-the-go while doing it. All of these behaviors factor into mobile survey design and shape the respondent experience to be a positive one.

In-app delivery

Survey tools, like Pollfish, integrate directly with mobile apps through developer partnerships, meaning that surveys are presented to respondents who are already engaged in an app on their device and are not sent through a different site or channel to complete the survey. The fewer clicks a respondent needs to take to get to the survey, the more likely they are to complete it. 

User engagement

Because mobile is with us wherever we go, it’s harder to predict where respondents will be when taking the survey. This improves data quality, as the respondent is taking the survey during a regular part of their day and providing answers that are more aligned with their gut-reaction, rather than offering answers that sound like what the survey wants to hear. However, this also means that respondents can become distracted or lose interest easily because of their surroundings. A well-designed mobile survey experience will include the ability to offer many question types, attach media, and an appropriate incentive to keep them engaged. It is important to note that although there should be many question types are available to use, it’s not necessary to make the survey long. Mobile is best used to gather quick, quantitative information. 

Mobile survey design tips

While your tool should be designed for the best mobile experience possible for respondents, it’s also important to think about the impact you have on the respondent experience as the creator of the survey questions.

Limit the number of questions

When creating a questionnaire, remember that shorter surveys are best. Attention spans are short and respondents are on the go, so it’s best to keep the questionnaire fewer than 25 questions long.

Short question length

With that in mind, the questions themselves should also be short, clear, and concise. If there is a need to convey a more complicated idea, present the information as a description at the beginning of a series of questions, or break it into smaller pieces. Consider using multimedia, such as images, or a sound clip if your question requires too much text to explain. 

Mix up question types

Mobile survey tools often present different question types that you can include. While some of the most popular (and easiest to use on mobile) include those that can be selected by tapping or selecting from a drop-down menu, rating scales and sliders can be used to break up a longer survey and keep it engaging. 

Keep a mobile mindset

You don’t need to be a UX designer to consider how someone would interact with a question using a smartphone. For example, it’s easier for a person to tap a response rather than to drag, draw, or type, so use open-ended questions and other actions that require effort from respondents sparingly.

App integration of mobile surveys

A “mobile-optimized survey” can mean one that formats for the best respondent experience on a mobile website or natively within an app that the respondent is using. 

However, in-app mobile survey delivery offers specific benefits that mobile web-optimized surveys do not. 

When a survey tool works directly with the app developer to include a line of code in the app, surveys are a seamless part of the mobile experience. Respondents can complete the survey in the app rather than being taken to a new window. This also allows for the collection of a unique identifier, so a profile can be based on demographic information that respondents provide upfront and can be used to target them later. This saves on the need to ask demographic questions at the beginning of every survey, and questionnaires can remain shorter. 

In-app survey delivery is able to work because of the publisher partnership model, which is a benefit to the app developers as well. Developers monetize apps by serving mobile ads to their users, but mobile surveys give developers the option of offering an alternative for an even higher return. Surveys are more engaging for their users, as they get to participate rather than simply consume content, and often there is often a perk associated with completing the survey that entices respondents to choose it over an ad. In-app mobile surveys are a great option for respondents, developers, and those who are looking for consumer insights. 

Pollfish, in addition to direct app integration, works with the app publishers to determine the incentives of the survey. This has an additional benefit of removing the survey company from the incentive selection, which can cause survey bias. Using app-specific incentives rather than cumulative points or monetary benefits also greatly reduces the cost of gathering responses, meaning that Pollfish is able to provide a significantly more affordable method of gathering data than other tools. 

To use mobile surveys effectively, market researchers must consider the respondent experience. It’s critical to offer a survey in the clearest, most concise, and effortless way possible to collect responses at scale, which differs in a mobile survey from that of a desktop form, pen-and-paper questionnaire, or a focus group. The user-experience is for more than just the creators of a survey on the platform, the respondent experience also plays an important role in the quality of data to come out of it. 

Want to spread the word about mobile & online survey design? Check out the Respondent Experience infographic!