Product management basics: conducting user interviews

In a world where quantitative data drives so many product decisions, interviewing users is still an important tool for understanding your users and get qualitative insights.

I invited Yael Oppenheim, a market and a user research consultant, and the founder of FitMyTime (a platform for online live personal fitness and yoga classes) for coffee.

Yael was kind enough to share some of her tips about conducting user interviews, and our conversation ended up with me interviewing her for the mobile spoon 😉. Is that a remarkable journalism work or what?


So here's an interview about how to conduct user interviews:

When your product exists, and you’re talking to your users, there’s a clear set of rules to make sure you eliminate biases and get genuine answers. But how do you do this when the product does not exist yet, and there aren’t any users to talk to?

You would think that doing user interviews without having an actual product (nor actual users) would make your work more difficult, because you can’t ask questions about something that doesn’t exist, right? But actually, doing user interviews before you even write one line of code or even prototype can save you a lot of time and money, and it will definitely help you make sure you are solving THE RIGHT IT, rather than solving IT RIGHT.

[Me: let me tweet it for a second: "Make sure you are solving THE RIGHT IT, rather than solving IT RIGHT" - Click to Tweet]

Doing user interviews before you build your product is part of a discovery phase, and what you want to do in that phase is to deeply understand your potential users’ pain. That’s it.

You can do that in several ways, and one efficient way to do it is by talking to people whose problems you want to solve. You want to generate a deep understanding of the problem and the way it affects people's lives.

You would also want to learn how they are currently solving this problem (and if they don’t solve it - then it is not a problem! Not a meaningful one at least). 

The solution/your product is not relevant to the discovery phase. All that is important when doing user interviews during the discovery phase is to learn about the problem area.

The same set of rules that apply to existing customers also apply to potential customers, and we should avoid questions that may create bias. For example: if you want to build a new navigation app that will compete with Google Maps or Apple Maps, you should ask potential users (i.e ones who commute a lot) to tell you about their work and about their driving/commute experience. Find out what annoys them, what problems are they having, what apps are they currently using and what they like/don’t like about them. You really don’t need to tell them what solution you have in mind, because that will distract you from the important thing - the problem they are having.

How do you find the right people to talk to?

That depends on the type of people you need to talk to, or in other words, whose problems you are trying to solve. If it’s salespeople, for example, you can ask your friends if they know any salespeople and ask for referrals, and then ask each salesperson you interview to refer you to a colleague of theirs.

There are many ways to reach all kinds of people. If we are looking for people with a specific interest, like gardening, or a specific problem, like back pain, then there’s a good chance they have a Facebook group, a Subreddit or other kind of forum they engage with. That can also be a good way to recruit people for interviews, however, you should ask for permission first because not all groups admins will allow this.

When searching for participants, you can make a criteria list that will help you screen them. For example, if you know you only want to interview people who drive at least 5 hours a day, you can create a screening survey with some questions that will help you find the right people.

[Me: to add my 2 cents, LinkedIn is a great place to find people based on their profession or companies they work for. Dedicated events are also a nice way to find relevant people to interview].

How do you convince them to talk to you?

That can be very easy or very hard... depending on who you’re trying to reach.
In general, if you explain that you are trying to help them or that you are working on a solution to some problem that they suffer from, or that you are developing something they can otherwise benefit from, you will have better chances to get their attention than if you just ask to interview them “for a research”.

For example, when I interviewed yoga and fitness trainers for FitMyTime, I told them that I was trying to help people who want to exercise with a personal trainer, but can’t find one that fits their needs and schedule in their area. Those people were potential customers for the trainers I spoke to, so they were very interested in talking about their experiences as personal trainers.

[Me: I would also add bribery, threats, kidnapping, impersonating to a recruiter, or simply polishing your pleading skills].

How many interviews are “enough”? And what’s the conversion rate from contacting people to an actual interview?

Well, typically you would want to interview 12-15 people, at least.
It really depends on the size and variety of your target audience. If you are building a marketplace for beauty services on-demand, for example, you’ll need to interview 2 types of users, beauticians, and clients.

The conversion rate depends, again, on the type of people you are trying to reach. If your potential users are students, for example, you may have around 70%-80% conversion rate (especially if you can pay for their time, which can also create some bias, but that’s another issue).

If, in contrast, you are trying to interview brain surgeons or VC investors, you can expect a much lower conversion rate. But again, you don’t need dozens of interviewees to get started, and once you have the first few you can ask them to refer you to others. It is important to remember that user research is a continuous process, and you need to talk to (potential and existing) users all the time.

[Me: I've been experiencing a lower conversion rate for my cold emails, but I guess it has to do with the nature of users and the quality of the copy. A good tip would be to try various subject lines, and stick with the ones that convert better].


A random break for some shameless promotions: 

• 84 cognitive biases that will help you design better-converting products - part 1
• 84 cognitive biases that will help you design better-converting products - part 2
• 5 mistakes product managers still make 
• The good, the bad, and the ugly truth about early adopters

We all know it’s important to avoid theoretical questions. How do you validate a need or a pain without getting into theoretical questions?

Yes, it can be dangerous to ask hypothetical questions because then what you’ll get are answers that may have nothing to do with your interviewees’ real day to day life.

Alberto Savoia, one of Google’s first employees beautifully articulated it:

“It’s much easier to get people to open their mouth than to open their wallets” 

- Click to Tweet

[Me: true, here are 10 lessons learned from asking our users to open their wallets...]

So, for example, if you want to build a dating app for dog owners, you don’t want to ask dog owners if they would use this app if you built it for them, because there is a good chance they will say they would definitely use it, but as we know, there are countless apps that people downloaded but never used more than once.

If you ask people to answer a hypothetical question, they might try to be nice to you and give you the answer you want to hear. You could, however, talk to people about their dating experience and see if they mention that finding other dog owners is a serious problem for them, that existing dating apps don’t really solve. If it comes up naturally in the conversation, you may have got something here.

Another thing about theoretical questions is that they invite people to predict what they will do in the future, which humans do not excel in doing. So asking people to predict whether they plan to consume less sugar and do more exercise means nothing more than wishful thinking, right? That’s because, in the future, we are all fit and healthy. The same will happen when you ask people if they would use a health app that will give them daily menus (based on machine learning and AI of course).

They might actually imagine how they are going to love that genius app you invented and use it till the end of days. However, if you ask them about the last time they used an app that helped them to keep a healthy lifestyle and they tell you that they tried many apps but nothing worked and that the only thing that worked for them was hiring a professional dietitian, then you know that a solution that includes personal human guidance may be a less risky direction then an app without human guidance.

[Me: it reminds me of the rise and fall of our local neighborhood running app that was prevented by piloting the idea with WhatsApp. In theory, everyone was excited about it and joined the group. In practice it worked exactly 0% of the time].

Only then you should begin to think about prototyping or testing your idea.

Some people would say that you can validate by simply creating an MVP, without talking to users at all. However, if you are wrong, and your MVP fails, you wasted time that you could have saved by first understanding the problem/pain and only then start to think about different potential solutions. And just to be clear, the risk here is not only wasted time and money, it’s that your solution may have been wrong, but that doesn’t mean there is not a problem to be solved. You just need to better understand it.

So, to answer your question, you can validate a need or a pain without asking a theoretical question by looking for evidence that there’s a problem to be solved, instead of looking for evidence that our idea is a good solution.

10 successful product leaders share their tips for creating successful products in 2020

How do you get a confirmation on a proposed solution without leading the user you are interviewing?

Confirmation is probably something we can only get when we put our solution for use. That’s the real test. The purpose of user interviews and user research, in general, is not to predict whether something will work or not, it is only meant to reduce risk, move the product forward and help to make informed decisions rather than relying on guts and hunches.

What this means is that there's definitely a chance that even if the research says that something won’t work, it may actually turn out to be a success, (and vice versa) and those things have happened in the past. But, again, that’s a risk management issue.

So assuming we validated that there’s an actual problem to be solved, now we would like to test our solution. At that point, we can test our solution using different MVP techniques like a landing page or a concierge MVP, to see initial traction before we build anything.

The basic idea is to find a creative way to “fake it before you make it”.

While you can’t get a confirmation for your solution in an interview, you can confirm that there is a problem that people are struggling with, and you can confirm how they are currently solving the problem, and you can confirm that the current solution/s are messy and not good enough.

[Gil: wait, this one deserves another tweet...]

You can’t get a confirmation for your solution in an interview, but you can confirm that there's a problem that people are struggling with, you can confirm that they are trying to solve it, and you can confirm that their solution/s are messy and not good enough 

- Click to Tweet

Once you nail the details of the problem and pain, the chances that you will be able to build the right solution (as opposed to a solution) are significantly better.

What are your top 3 tips for conducting a successful interview?

1. Ask people to tell you about their problem, not to give you their opinion about your solution. For example, it’s better to ask “how do you manage your meals while you are at work?” than “would you use my healthy food delivery app that will help you find the best healthy food in your area so you will never be tempted to order junk food”?
2. Ask open-ended questions that enable people to elaborate, and don’t ask Yes/No questions that force them to give a potentially biased answer. For example, instead of asking “Are you finding it hard to meet deadlines?” It’s better to ask: “Can you tell me about a time when you missed a deadline”?
3. Ask people about specific actions they recently did, not about their general habits. For example, instead of asking “how many times do you usually cook?” it’s better to ask: “how many times did you cook last week”? This phrasing will help you get a more accurate indication of people’s actual behavior.

Do you have an example of a user interview that completely changed the perception of the problem/solution? 

There's a famous story about the Heinz ketchup bottle evolution. The original, classic bottle was this glass bottle and you had to shake and smack it a few times to get the ketchup out. Heinz sent a team of researchers to talk to customers and to observe how they were using ketchup at home. They noticed that small children, who loved ketchup, were not able to use the bottles, because it was difficult for them and when they tried they got too much ketchup out of the bottle, which their mothers really didn’t like. So, most of the time the mother would give the kids their ketchup and she was controlling the amount. Heintz then realized they should give kids control and that’s how the plastic, squeezable bottle was born.

In this case, the insight was gained while observing usage, not only by talking to users but in some cases, you can do interviews while observing people in their natural environment, which is ideal for a researcher because then you can see things that the user may not necessarily be aware of because it’s just part of their habit and they are not giving it a lot of thought.

[Gil: I remember a story where a certain user of a scheduling system, kept resizing appointments on the Gantt, breaking some logic rules, sending notifications to the clients, and creating a sick chain reaction, just because he couldn't see the title of the appointment properly and didn't know there's a zoom-in option... 🤨. A true story...]

So to continue with that notion, what else can be done do if you can’t reach out to your users due to a budget or privacy limitations? 

Interviewing is only one of several ways to learn about users’ needs and problems. Observation is a great way to learn about users’ behavior, habits, and problems.

The problem with observations is that it takes a lot of time and resources and it’s only relevant for specific situations because you can only observe people in the public sphere, unless you do what Heinz did, and you observe people at their homes or offices or wherever they encounter the problems you are trying to solve.

Of course, when people are being observed, they might change their behavior and not act naturally, which is something we need to take into consideration. And if you wish to observe people in a public place, you might be seen as a stalker or raise suspicion, so that can be challenging.

However, there is more than one way to observe what people do. In the past few years, I have done a lot of user research that is based on social data analysis.

Before the internet, it was really hard to learn about needs without directly talking to people or observing them, but now, people constantly document their lives and express their wants and needs on Facebook, Twitter, Reddit and so on.

When it comes to products and services, you can also learn a lot from reading reviews and also from listening to the way people talk about what they like or don’t like. There is an abundance of data, and there are tools that help collect and make sense of this data.

The great thing about social data analysis is that it enables us to learn about people’s needs without having to ask them directly, because we can just “listen” to what they say online, in an unsolicited manner.

Also, the option to tap into so many discussions on social media enables us to get large quantities of qualitative data, which is very hard to get otherwise. This is a relatively new approach to user research and it has its caveats, of course. But even as a product manager/designer, you can sometimes find forums or groups where people talk about the problem you’re trying to solve, and you can find a lot of insights there.

Intercom, for example, found out that users were using the map feature to show potential investors their clients around the world. That completely changed their roadmap and their approach to this feature, because this was obviously not the original purpose of the map feature.

When Intercom understood this need, they redesigned this feature to enable their customers to show off and brag about their clients in front of investors in conferences. That is something that would probably not come up in an interview.

Recap

I had fun! Yael, thanks for taking the time to answer all my questions - it was great!

Here are some key points followed by a summary image: 

1. User interviews are not designed to confirm a solution, but they confirm a problem
2. A problem that isn't being solved is probably not that big of a problem
3. Ask open-ended questions and be ready to dig deeper with follow-up questions 
4. Don't: 
1. Don't ask people what they want
2. Don't' ask people what they would do
3. Don't ask yes/no questions 
4. Don't make assumptions 
5. Don't ask people about their opinion 
5. Do: 
1. Act normal. Don't be a weirdo. 
2. Ask open-ended questions 
3. Ask people about their recent activities 
4. Ask people about the last time they experienced what they say they did
5. Dig deep with follow-up questions

- - -

Yael Oppenheim is a market and a user research consultant. She helps product teams conduct lean market and user research for their products and services, gain user insights by fusing quantitative and qualitative methods and drive product outcomes that create value for their users in margin enhancing ways. Yael is also the Founder of FitMyTime - a platform for online live personal fitness and yoga classes.

- - -

Before you leave, check out this collection of essays about how to become a better product manager - the best of the mobile spoon. 

Or subscribe to the newsletter and get new posts directly to your inbox.