If you want to increase conversions, you have to figure out who exactly is your primary target audience, what they want, what matters to them and what are the sources of friction for them. If you say your target audience is “pretty much everybody” or “anyone interested in my services”, you don’t have much of a chance to boost conversions.
Qualitative research is mostly about learning who the customers are, what they want, the language they use. This is critical for copywriting, understanding friction, learning what matters to them about the products you sell, and so on.
Conversions are all about relevancy — if what you offer and how you present it matches their state of mind, you have gained a customer. If your customer is “everybody”, you’re making it extremely difficult for yourself — nobody will identify with “everybody”.
Whom to survey
Survey people who still freshly remember their purchase and the friction they experienced in the buying process. Only talk to your recent first-time customers (who have no previous relationship or experience with you that might affect their responses).
You want to filter out repeat buyers or people who bought a long time ago. If you ask somebody who made the purchase 6 months or more ago, they have long forgotten and might feed you with false information.
How many people to survey
While best online surveys are qualitative (open-ended questions), we still need a good number of responses in to get an adequate overview. If you only survey 10 people some loud voices can skew the picture, and you it’s easy to identify false patterns.
I’ve found that the best quantity is somewhere between 100 and 200. You don’t need more than 200 as the answers tend to get repetitive, and don’t offer additional insight. Remember — this is a qualitative survey, not quantitative (like opinion poll). Any less than 100, and there might not be enough answers to draw conclusions from.
If you have less than 100 people who recently bought from you, then you do with what you can get. 10 responses is better than zero.
Analyzing survey responses
First off lets be clear: there is no general consensus among qualitative researchers concerning the process of qualitative data analysis. There is no single right way to go about it.
What I’m telling you here is a process that has worked for me and many of my CRO peers + is advocated by some of the researchers.
It’s all manual labor.
- Be clear about the goals and what you are looking for
- Conduct an initial review of all the information to gain an initial sense of the data.
- Code the data: organize it into some manageable form. This is often described as ‘reducing the data’, and usually involves developing codes or categories (while still keeping the raw data)
- Interpret the data.
- Write a summary report of the findings.
While these are steps 1 to 5, I want to stress that the process of qualitative analysis is not a linear but rather continuous and iterative. It is perfectly normal and expected that you jump between all these steps, go back and forth.
Be ready to spend at least 4 hours on this, or even a couple of full working days. Don’t be afraid to put in hours to find insights.
Our main goal is to learn about customers. Typically we’re seeking to learn the following:
- Who these people are? What are the common characteristics? Can we form some hypotheses about different customer personas?
- What are the problems they are solving for themselves? We can use this in our value proposition when we state the problem we’re solving.
- What’s the voice of the customer like: how they word things? Your website has to speak the same language your customers do. Notice how they describe the problem, the solution, the desired benefits.
- What are the main sources of friction: doubts, hesitations, unanswered questions? Once we know this, we can take action to reduce the friction.
- How would they like to buy?
- Do they comparison shop? How much? This is important — if they shop around a lot, we need to stress more on our unique benefits and need to be visibly better/different from the competition.
- Any insights about their emotional state?
2. Initial review
In this phase, you go in and look at the responses question by question. Some questions can be grouped together (doubts & hesitations and unanswered questions are both about friction, and “who are you” and “which problem were you solving for yourself” are both about customer personas), and thus looked at together.
The goal here is to identify trends and patterns and create a “code” for each trend. The code is usually a word or short phrase that suggests how the associated data helps us reach the goals we set in the previous steps. Make sure you write the codes down!
Coding enables you to organize large amounts of text and to discover patterns that would be difficult to detect by reading alone. Codes answer the questions, “What do I see going on here?” or “How do I categorize the information?”
In the next phase, you go in and attach codes to as many responses as you can.
NB! Beware of your bias. It’s very human to identify a couple of trends right away (at least a perception of a trend), and then only start looking for information that proves the trend while ignoring everything else. Know that this will happen to you, and self-correct when you become aware of it. If needed, take a break and come back to the data the next day. Or better yet, have a second pair of eyes
NBB! It’s also typical to only pay attention to the first 50 or so results and then skim over the last 150. First responses are in no way more important than later ones. If you get tired and notice that you start skimming over the responses, take a break and come back to it later.
Now that you have a list of codes, go back and attach codes to as many responses as you can (significant part of the data). Not all responses can be labeled. It’s perfectly fine to tweak, add and eliminate the codes as you get a better grasp on the data at hand. Eliminate less useful ones, combine smaller categories into larger ones, or if a very large number of responses have been assigned the same code, subdivide that category.
The goal is to link elements of the data that are conceived of as sharing some perceived commonality.
For instance, I had a client whose product was ” vegan healthy meal plans”: weekly grocery shopping list and recipes for each breakfast, lunch and dinner for 7 days.
When I read through the answers the first time, I noticed that there are 3 typical use cases:
- Busy mom — too busy to think about what to shop and what to cook
- Overweight or sick people — want to get healthy by following the meal plans
- Vegan / people with celiac disease — people who bought it because of the gluten-free and vegan thing
So these 3 became my codes — during the 2nd reading, I went in and added comments / notes “busy”, “overweight” or “vegan”. I counted the number of responses per code to get an idea of the distribution.
This influenced how I prioritized the order of content on the sales page.
4. Interpret the data
- Now that you’ve read the data so many times, what are the patterns you’re seeing? There usually things that stand out.
- Write down what you can about hypothetical personas (as many as you can spot)
- Count how many responses per code you have to prioritize issues
5. Summary report
Write down the key learnings (your memory is not as good as you think) to always keep them at hand for formulating hypotheses (when comparing to other sources of data). It’s also essential for your teammates and clients.
Voice of the customer
Besides identifying trends, pay attention to their language. How do they phrase the problem? Often I copy and use the exact wordings from a survey answer in a value proposition or other key part of the website copy. It tends to works extremely well.
Sometimes it helps to generate word clouds from the responses. Most common words will stand out, and can serve as additional insight. Note that this is NOT as useful as reading actual responses.
Here’s a word cloud of a survey I did for a Paleo diet website, and the goal of the survey was to figure out “what people want”. Look at the cloud, and see if you get any insights (without knowing anything else)?
Can you name 3 things that people seem to want?
If you’re thinking that…
- their goals seem to be about weight loss, and they want to accelerate the process,
- they want specific how-to guides,
- and recipes for a healthy diet,
… you’d be spot on.
Now imagine if you could compare the word cloud with the actual form responses — it could serve as (in)validation to the hypotheses you have until now.
Here’s another Word cloud, formed from the answers to question “What are you using our service for?”. The goal of the question was to uncover needs and user intent, so we could adjust the value proposition and sales copy accordingly.
The service in question is about healthy 7-day meal plans.
So — any thoughts about what they’re looking for in this service?
Stuff that seems to be in your face:
I would now go compare these with my notes, and if needed go back to the raw responses and look up what’s going on there with the vegan stuff and husbands.
Tools to use
Word of caution: never use word clouds as a way of analyzing the surveys without actually analyzing the responses. You’ll start to make stuff up. It’s mostly meant to summarize and double check your findings — if there’s a large keyword you don’t have a code for, might be a good idea to go back to the survey and look up responses containing that keyword.
Tools to use for conducting surveys
Full disclosure: I’m taking the CXL Growth Marketing Minidegree as part of a scholarship, on the basis that I write one review post per week, over twelve weeks. If you’re interested in reading a really in-depth review of the Minidegree, keep an eye on my Medium page over the next 3 months. 🙂