Conducting Customer Surveys

If you want to increase conversions, you have to figure out who exactly is your primary target audience, what they want, what matters to them and what are the sources of friction for them. If you say your target audience is “pretty much everybody” or “anyone interested in my services”, you don’t have much of a chance to boost conversions.

Qualitative research is mostly about learning who the customers are, what they want, the language they use. This is critical for copywriting, understanding friction, learning what matters to them about the products you sell, and so on.

Conversions are all about relevancy — if what you offer and how you present it matches their state of mind, you have gained a customer. If your customer is “everybody”, you’re making it extremely difficult for yourself — nobody will identify with “everybody”.

Whom to survey

Survey people who still freshly remember their purchase and the friction they experienced in the buying process. Only talk to your recent first-time customers (who have no previous relationship or experience with you that might affect their responses).

You want to filter out repeat buyers or people who bought a long time ago. If you ask somebody who made the purchase 6 months or more ago, they have long forgotten and might feed you with false information.

How many people to survey

While best online surveys are qualitative (open-ended questions), we still need a good number of responses in to get an adequate overview. If you only survey 10 people some loud voices can skew the picture, and you it’s easy to identify false patterns.

I’ve found that the best quantity is somewhere between 100 and 200. You don’t need more than 200 as the answers tend to get repetitive, and don’t offer additional insight. Remember — this is a qualitative survey, not quantitative (like opinion poll). Any less than 100, and there might not be enough answers to draw conclusions from.

If you have less than 100 people who recently bought from you, then you do with what you can get. 10 responses is better than zero.

Analyzing survey responses

First off lets be clear: there is no general consensus among qualitative researchers concerning the process of qualitative data analysis. There is no single right way to go about it.

What I’m telling you here is a process that has worked for me and many of my CRO peers + is advocated by some of the researchers.

The process

It’s all manual labor.

  1. Be clear about the goals and what you are looking for

While these are steps 1 to 5, I want to stress that the process of qualitative analysis is not a linear but rather continuous and iterative. It is perfectly normal and expected that you jump between all these steps, go back and forth.

Be ready to spend at least 4 hours on this, or even a couple of full working days. Don’t be afraid to put in hours to find insights.

1. Goals

Our main goal is to learn about customers. Typically we’re seeking to learn the following:

  • Who these people are? What are the common characteristics? Can we form some hypotheses about different customer personas?

2. Initial review

In this phase, you go in and look at the responses question by question. Some questions can be grouped together (doubts & hesitations and unanswered questions are both about friction, and “who are you” and “which problem were you solving for yourself” are both about customer personas), and thus looked at together.

The goal here is to identify trends and patterns and create a “code” for each trend. The code is usually a word or short phrase that suggests how the associated data helps us reach the goals we set in the previous steps. Make sure you write the codes down!

Coding enables you to organize large amounts of text and to discover patterns that would be difficult to detect by reading alone. Codes answer the questions, “What do I see going on here?” or “How do I categorize the information?”

In the next phase, you go in and attach codes to as many responses as you can.

NB! Beware of your bias. It’s very human to identify a couple of trends right away (at least a perception of a trend), and then only start looking for information that proves the trend while ignoring everything else. Know that this will happen to you, and self-correct when you become aware of it. If needed, take a break and come back to the data the next day. Or better yet, have a second pair of eyes

NBB! It’s also typical to only pay attention to the first 50 or so results and then skim over the last 150. First responses are in no way more important than later ones. If you get tired and notice that you start skimming over the responses, take a break and come back to it later.

3. Codification

Now that you have a list of codes, go back and attach codes to as many responses as you can (significant part of the data). Not all responses can be labeled. It’s perfectly fine to tweak, add and eliminate the codes as you get a better grasp on the data at hand. Eliminate less useful ones, combine smaller categories into larger ones, or if a very large number of responses have been assigned the same code, subdivide that category.

The goal is to link elements of the data that are conceived of as sharing some perceived commonality.

For instance, I had a client whose product was ” vegan healthy meal plans”: weekly grocery shopping list and recipes for each breakfast, lunch and dinner for 7 days.

When I read through the answers the first time, I noticed that there are 3 typical use cases:

  1. Busy mom — too busy to think about what to shop and what to cook

So these 3 became my codes — during the 2nd reading, I went in and added comments / notes “busy”, “overweight” or “vegan”. I counted the number of responses per code to get an idea of the distribution.

This influenced how I prioritized the order of content on the sales page.

4. Interpret the data

  • Now that you’ve read the data so many times, what are the patterns you’re seeing? There usually things that stand out.

5. Summary report

Write down the key learnings (your memory is not as good as you think) to always keep them at hand for formulating hypotheses (when comparing to other sources of data). It’s also essential for your teammates and clients.

Voice of the customer

Besides identifying trends, pay attention to their language. How do they phrase the problem? Often I copy and use the exact wordings from a survey answer in a value proposition or other key part of the website copy. It tends to works extremely well.

Word clouds

Sometimes it helps to generate word clouds from the responses. Most common words will stand out, and can serve as additional insight. Note that this is NOT as useful as reading actual responses.

Here’s a word cloud of a survey I did for a Paleo diet website, and the goal of the survey was to figure out “what people want”. Look at the cloud, and see if you get any insights (without knowing anything else)?

Can you name 3 things that people seem to want?

If you’re thinking that…

  • their goals seem to be about weight loss, and they want to accelerate the process,

… you’d be spot on.

Now imagine if you could compare the word cloud with the actual form responses — it could serve as (in)validation to the hypotheses you have until now.

Here’s another Word cloud, formed from the answers to question “What are you using our service for?”. The goal of the question was to uncover needs and user intent, so we could adjust the value proposition and sales copy accordingly.

The service in question is about healthy 7-day meal plans.

So — any thoughts about what they’re looking for in this service?

Stuff that seems to be in your face:

  • vegan,

I would now go compare these with my notes, and if needed go back to the raw responses and look up what’s going on there with the vegan stuff and husbands.

Tools to use

Word of caution: never use word clouds as a way of analyzing the surveys without actually analyzing the responses. You’ll start to make stuff up. It’s mostly meant to summarize and double check your findings — if there’s a large keyword you don’t have a code for, might be a good idea to go back to the survey and look up responses containing that keyword.

Tools to use for conducting surveys

  • I absolutely love Google Docs forms. It’s simple and free. (But some don’t since it doesn’t have bells and whistles, and looks kind of plain).

Full disclosure: I’m taking the CXL Growth Marketing Minidegree as part of a scholarship, on the basis that I write one review post per week, over twelve weeks. If you’re interested in reading a really in-depth review of the Minidegree, keep an eye on my Medium page over the next 3 months. 🙂

Data-driven Digital Marketer & Photographer|| Based in Lagos, Nigeria. Find me at linkedin.com/in/anita-princewill/