Surveys. We either adore them or despise them.
Why are they so polarizing? It is difficult to find a more misunderstood UX research tool than questionnaires. On one hand, surveys are frequently the go-to research approach for many product teams due to their minimal cost and time requirements. On the other side, we encounter qualitative researchers who despise surveys and say that the data acquired there is, at best, skewed or uninformative.
So, which side is correct?
We believe that striking a balance will yield the greatest outcomes. The purpose of this paper is to demonstrate how researchers working in product teams may accomplish this. As a starting point, we'll look at which practices should be avoided in order to prevent raising eyebrows. In the second half, we will discuss how we may improve our surveying techniques. Finally, we will show you some inspiring UX survey use examples. Let's get started.
"There's no such thing as a foolish question". This is a sentence we've all heard a million times.
Curiosity is a great quality in general. But ask any researcher, or anybody else whose job it is to collect accurate data from individuals. You name it: reporters, detectives, or psychologists. They will undoubtedly inform you that certain techniques of asking questions are more advantageous than others. Yet, before we get there, consider what occurs when questions are asked in an unfavorable manner.
1. Sending a random set of queries into the internet
You have some research questions to answer as a researcher. Yet, you lack the resources (such as time, incentives, and tools) to obtain them. Yet finding precise target groups for a thorough qualitative study might be time-consuming.
Therefore, why not just enter your study questions into a free survey program and distribute them to a large enough online group?
It is a simple, quick, and inexpensive approach to obtaining data. If you join any UX-related Facebook or Slack community, you won't have to wait long for a new poll to appear. It is usually followed by requests to fill it out.
Anyone who does it for free may do it. And very soon. The survey will obviously take no longer than 10 minutes. Maybe 15. But it is the limit. Pinky-promise.
Nevertheless, just as making a delicious risotto involves more than putting butter and rice in water, a list of questions thrown together at random would not provide a decent survey. It is preferable not to conduct research unless there is a clear emphasis on both the aims and the intended participants. So there you have it.
An example of an unstructured collection of survey questions Example of an unstructured set of survey questions - source: writer's own collection consider that for a moment. And be truthful to yourself.
Do you truly believe that any random sample of opinions results in equally sound product decisions? Consider gathering the facts in person. By phoning your neighbors at random. One of them is a surly adolescent. The other is a kind grandmother. Who is most likely to provide meaningful answers to your questions?
Although there is always the possibility that this grandmother will give useful information on TikTok's usability, her viewpoint is mostly simply a distraction. When anybody can fill out your survey, you take a tremendous risk.
Even if the sampled grandmother is a prospective target consumer... Are you certain she understands your inquiries?
As you sample a diverse group of people, the level of understanding of your questions will vary. This affects the dependability of your data. And we have not even touched on the fact that a random collection of questions might result in fluctuating motivation during the fill-in procedure. The arrangement of the questions influences how individuals respond.
The sort of questions asked has an impact on how individuals respond. Even the weather has an impact on how individuals respond to inquiries. Are you certain you considered all of this before making your survey public?
2. Fortune-telling with surveys
We all desire to forecast the future. Particularly while developing new products or services. Perhaps this is why one of the most prevalent errors we encounter in surveys is the broad use of conditional items.
These generally sound like "Imagine you're purchasing a rocket...", "How would you act if you become...", or "Would you appreciate this app if you were a new mother...". The worst part of these questions is that they result in presentable conclusions, such as "80% of ladies above 30 who reside in large cities would buy a rocket".
So, should we begin constructing these rockets? Not so quickly. Humans are highly skilled at envisioning the future. Especially ones that boost their self-esteem. But will they act on their fantasies? The outlook is no longer so rosy.
Through conditional queries, we will only obtain responses regarding their intents. It is recommended to avoid utilizing them if you are interested in real behavior. In surveys, an example of "brainstorming" questions Source: writer's personal collection of "brainstorming" survey questions
3. Unmoderated surveys used as interviews
"Surveys are not written interviews," we say again. We can't tell you how many times we've seen someone publish a list of open-ended questions and then call it quits. And once this happens, we as researchers feel as though something within of us dies. Surveys as a substitute for in-depth interviews - source own collection Trust me, we understand.
Conventional interviews might be difficult to perform at times. You might not have time to speak with everyone. You might not have enough money for incentives. If you don't want to deal with recruitment issues.
But first and foremost, is it acceptable to ask unassuming volunteers to perform your dirty job and answer extensive questions without compensating them? It's not just about the money, either.
Participants in professional, well-led user interviews have the opportunity to communicate with the researcher. They discuss their difficulties or wants and we, researchers, help them feel heard. It's a positive and fruitful experience.
This is not the case while filling out a faceless survey. Apart from ethical concerns, there is the issue of data quality. In the event of open-ended queries, you must just accept any response at face value. Even if they are more simple. Or it might simply be unclear.
Only after you start analyzing data do you realize how difficult it is to cope with dozens of replies of varying quality. At the end of the analysis, you will most likely regret employing surveys as interview substitutes. Surveys as a substitute for in-depth interviews, for example, Surveys used as a substitute for in-depth interviews - source: writer's own collection
Let's move on to calmer seas now that we've seen how not to run surveys. Let's look at what contributes to the creation of a legitimate, trustworthy, and informative survey. Most academics describe surveys as a series of questions (a questionnaire or numerous questionnaires together) obtained from your target population.
They are meant to evaluate attitudes, beliefs, specific behaviors, or their prevalence in those target groups and are then analyzed. Long definition, we know. But, in order to reach tremendous achievements, we must have firm foundations. Let us now examine what this entails in reality.
1. The right time for surveys
There are several sorts of research questions for which you should consider employing questionnaires. One of these common scenarios is when you need to know how frequently a given behavior or attitude occurs or how much a certain behavior or attitude occurs. You may go into further detail hereby categorizing your different categories of consumers.
For example, via surveys, you may measure how many of the visitors, who have visited your listings page have discovered that the listings are appealing to them. Alternatively, at a later stage of the trip, you may assess how many of your consumers recall a specific aspect from the listings page.
Besides quantifying quantity, surveys are also useful for determining how people feel about specific topics. This is not to be confused with the problematic practice of using surveys to investigate the whys of behavior.
Let us explain the distinction between them. It's really simple to describe how much you enjoy a pistachio ice cream or whether you prefer it over lemon ice cream. You adore pistachio ice cream and prefer it over lemon.
You probably likely remember how many times you've had pistachio ice cream in the last several weeks. It only takes a few seconds to recall something. Yes. Can you correctly choose answers from a drop-down list? Absolutely.
But first, tell me why pistachio ice cream is so delicious to you. In that circumstance, providing an accurate response becomes more difficult. Whether it reminds you of family vacations, perhaps you like cream-based ice creams, or perhaps your favorite ice cream store simply has a fantastic recipe.
Then again, who doesn't? How likely is it that you will fill out all of these questions in a survey? Especially one you got on your phone from a stranger? Not at all likely. As we can see, surveys are not a "bad" approach, but they must be used to answer the proper question. We obtain dense, emotive, and in-depth data from just a few folks.
But, there are instances when we need to assess attitudes and quantities on a wide scale. In certain instances, as demonstrated by the ice cream example, qualitative approaches would be a waste of time.
Furthermore, reaching a large scale might assist you in balancing your qualitative data. Yeah, we discuss confirmation bias in product discovery. As your team is making long-term product choices, surveys may help you confirm how much your consumers actually exhibit the focus behavior.
Before we continue, we'd like to point out that surveys may be used with quantitative methodologies as well. For example, you might utilize analytics to track the success of a homepage design. Yet, surveys are the most cost and time-effective approach for learning not just what individuals do but also how they feel about what they do.
2. Before your very first survey
You may be fortunate enough to enter product design with a solid quantitative background. Several of us have completed research-intensive programs. We're all aware of issues like how to develop impartial questions, data-cleaning strategies, and statistical methodologies in those circumstances.
Nonetheless, UXR is a vibrant field. It's possible that your skills lay elsewhere. In this instance, we do not recommend diving right into your first survey. Take the time to adequately prepare beforehand. Discover survey methodologies. That does not imply that you must return to university. Or that you should invest in pricey courses. There are various trustworthy online courses that may be completed at your own leisure.
Surveys are also covered in the IDF's quantitative training. These classes are an excellent method to develop your trade and gain confidence in the dependability of your findings. Just remember, you are not alone. Creating surveys is not a natural talent, but rather something that must be learned and practiced.
If we could just advocate for one thing, it would be this: do not be afraid to consult with other researchers who are more experienced in this field. Even if you are the only researcher in your organization, you may connect with others through Facebook groups, Slack channels, or LinkedIn. In our experience, UX research is an area where most individuals understand the difficulties of getting started and are typically eager to assist.
We hope you're delighted to add surveys to your toolset by now. And you should be: surveys are an excellent instrument for quantifying qualitative insights, gathering constant and low-cost feedback on people's opinions, and assisting in the prioritization of crucial choices.
That is when surveys are correctly conducted. We hope that after reading this essay, you will be more confident in deciding when you should and should not use them.
Recall that the only thing worse than going into product development blindly is going in with incorrect ideas. To assist you further, we will provide a detailed step-by-step tutorial for creating your own surveys.
At the same time, do not stress too much. Improve your craft, get advice from more experienced researchers, put your newfound knowledge into practice, and do it better the following time. Have fun while you're at it. Keep an eye on things.
Now survey-rich designs do not come with a DIY (Do It Yourself) Kit, do they?
Don’t worry, we’ve got you covered!
Dezy It has everything you were looking for. Our vast knowledge and professionality enable us to deliver top-notch well researched and put-together designs.
Namely, with us, you will get:
Personalized approach
Transparent workflow
Extensive portfolio
Fast time to market
Different cooperation models
A talented pool of IT professionals
Sounds too good to be true?
But all these benefits are waiting for you at Dezy It.
So, don’t waste any more time, contact us, and we will not let you down!