Pros and Cons of Conducting User Interviews

You have had a green light from your stakeholders to conduct some user research for your design project, and you think user interviews might be the way to go. After all, they seem straightforward, and interviews are one of the most widely used user research methods — so, they should be a safe choice. But, as with all research methods, there are pros and cons of using user interviews to gain insights for your design project, so knowing how to use interviews correctly is nothing short of vital. Here, you will learn in what situations user interviews are appropriate and insightful, and when you are better off choosing another research method.

Table of contents

What Are User Interviews?
Pros and Cons of User Interviews
When to Conduct User Interviews
The Take Away
References & Where to Learn More
Images

What Are User Interviews?

Interviews can be a great way to empathize with your users because interviews can give you an in-depth understanding of the users’ values, perceptions, and experiences. They allow you to ask specific questions, while remaining open to exploring your participants’ points of view. They are also often combined with other user research methods, such as usability tests or surveys, so as to gain deeper insights into objective results by asking a user about them and to elicit the user’s subjective opinion on products or interactions.

We are aware of interviews from many different contexts, from magazines to job interviews, but the term “user interviews” often refers to semi-structured qualitative interviews, which is a research method with roots in the social sciences. As the term implies, semi-structured interviews are somewhat structured in that you prepare a set of topics you would like to cover during the interview, but still open enough that you can follow leads in the conversation and change the order of topics.

In this video, professor of Human-Computer Interaction at University College London and expert in qualitative user studies Ann Blandford describes what characterizes a semi-structured interview.

Show Hide video transcript

00:00:00 --> 00:00:30

In a sense, the answer is in the expression 'semi-structured'. So, a completely structured interview is a *conversation where all the questions are pre-scripted* and very often the answers are *closed answers*, so like an option 1 from 5 or a closed question like 'Yes' / 'No'.
00:00:30 --> 00:01:02

And if you're going to do that kind of interview, really it's very much like a *survey*, and you might almost do it – it might be better as a written survey rather than as an interview. At the other end of the spectrum, you just have a *conversation, a chat with somebody*, which might or might not be on a particular topic or might range across a whole spread of topics and cover all sorts of material not necessarily with any particular structure and not with a particularly obvious purpose to it.
00:01:02 --> 00:01:32

*Semi-structured lie between these extremes*, i.e. there is some kind of structure to it; you typically have a *topic guide* or a *semi-structured interview script* that determines the kinds of topics that you're going through and an initial planned order with which you'll cover those topics. But it's also free form in that people are welcome to give long answers, like the answers I'm giving you now.
00:01:32 --> 00:02:02

And you can elaborate on questions; so, if somebody says something particularly interesting that you hadn't anticipated, you might follow up on that. And indeed sometimes people answer the topic that you'd thought of as being the fifth topic – they might introduce that when you thought you were still talking about the second topic; and so, it kind of *free-flows* to some degree and feels natural. But as an interviewer, you have an agenda;
00:02:02 --> 00:02:33

you have a set of topics that you want to cover; you perhaps even have some detailed plans of how you're going to ask some of the questions so that you get through the topics that *you* want to cover even if participants maybe go off topic at times or – you know – divert into other things. So, that's roughly what a semi-structured interview is. It's a *conversation*, but it has a set of *topics* that you want to cover
00:02:33 --> 00:03:09

and is *organized* such that you're reasonably confident that you will cover those topics, but perhaps in a way that is responding to the participant and their interests and to some extent their agenda as well, as well as that of the interviewer, but it's not a completely fixed structure where you're not allowed to deviate from what you've planned ahead of time. In HCI, I think they're the most common form of interview because
00:03:09 --> 00:03:34

we typically have an agenda; we're concerned with finding out from users or from potential users about the requirements for future designs or about the way that people do their work so that we can design new technologies to support that work or that activity better. Or maybe there's already an existing technology that you're testing,
00:03:34 --> 00:04:00

and so you want to know specific things about what works well, what doesn't, and so you have an agenda as an interviewer; it's not just a casual conversation. But you do want to know a set of things about that thing. But you also want to hear what the user has to say; you want to listen to them, especially when they say things that are *unexpected*,
00:04:00 --> 00:04:11

as well as finding out the answers to questions that you've asked. So, as interviews go, they are – I would say – the most common form used in HCI.

When you conduct semi-structured interviews, you have an interview guide with the questions or themes that you want to talk to the user about, but you are free to change the order of questions or to explore different topics that may arise during the interview. The advantage of the semi-structured interview approach is that you can define a predetermined set of topics that you know to be relevant to your project before you conduct the interview, but you can also explore topics that you had not previously thought relevant. This makes the method especially suitable for creating insights in design projects — because design projects are not usually completely exploratory; instead, they aim to find solutions to specific problems or challenges.

Man being interviewed on camera in the woods. — In many ways, an interview is like an everyday conversation, but it’s important to be aware that a good interview requires consideration and structure.

If you google “user interviews,” you might encounter blogs and other web resources which state that interviews are an easy way to create insights for your design project because conducting an interview is like having a conversation with your users — and that doesn’t require a lot of preparation. While interviews obviously share similarities with everyday conversations, a good interview requires preparation and careful consideration on the part of the interviewer. Steve Portigal, founder of Portigal Consulting and author of the book Interviewing Users, states that while interviews superficially resemble more informal social occasions (you meet someone at their home, they offer you coffee, you comment on the weather, etc.), it’s important to be aware of how to ask questions and how to listen in order to gain valid insights into your participant’s life and experiences:

“To learn something new requires interviewing, not just chatting. Poor interviews produce inaccurate information that can take your business in the wrong direction. Interviewing is a skill that at times can be fundamentally different than what you do normally in conversation. Great interviewers leverage their natural style of interacting with people but make deliberate, specific choices about what to say, when to say it, how to say it, and when to say nothing. Doing this well is hard and takes years of practice.”
— Steve Portigal, author of Interviewing Users

So, interviewing is a skill, but it’s a skill that can be learned, and the first step involves knowing when to use interviews and when not to use them.

Pros and Cons of User Interviews

“What users say and what they do are different.”

— Jakob Nielsen, Usability Expert and co-founder of Nielsen Norman Group

User interviews can be very informative and helpful, but only if they are used correctly and for the right things. It’s important to know what you can expect to get out of interviews and what you shouldn’t expect to get out of interviews.

In this video, Ann Blandford explains what you can get out of semi-structured interviews and what you shouldn’t expect to be able to do.

Show Hide video transcript

00:00:00 --> 00:00:35

So, semi-structured interviews – well, any interview, semi-structured or not, gets at people's perceptions, their values, their experiences as they see it, their explanations about why they do the things that they do, why they hold the attitudes that they do. And so, they're really good at getting at the *why* of what people do,
00:00:35 --> 00:01:02

but not the *what* of what people do. That's much better addressed with *observations* or *combined methods* such as contextual inquiry where you both observe people working and also interview them, perhaps in an interleaved way about why they're doing the things that they're doing or getting them to explain more about how things work and what they're trying to achieve.
00:01:02 --> 00:01:32

So, what are they *not* good for? Well, they're not good for the kinds of questions where people have difficulty recalling or where people might have some strong motivation for saying something that perhaps isn't accurate. I think of those two concerns, the first is probably the bigger in HCI
00:01:32 --> 00:02:00

– that... where things are unremarkable, people are often *not aware* of what they do; they have a lot of *tacit knowledge*. If you ask somebody how long something took, what you'll get is their *subjective impression* of that, which probably bears very little relation to the actual time something took, for example. I certainly remember doing a set of interviews some years ago
00:02:00 --> 00:02:32

where we were asking people about how they performed a task. And they told us that it was like a three- or four-step task. And then, when we got them to show us how they did it, it actually had about 20, 25 steps to it. And the rest of the steps they just completely took for granted; you know – they were: 'Of course we do that! Of course we—' – you know – 'Of course that's the way it works! Of course we have to turn it on!' And they just took that so much for granted that *it would never have come out in an interview*.
00:02:32 --> 00:03:11

I mean, I literally can't imagine the interview that would really have got that full task sequence. And there are lots of things that people do or things that they assume that the interviewer knows about, that they just won't say and won't express at all. So, interviews are not good for those things; you really need to *observe* people to get that kind of data. So, it's good to be aware of what interviews are good for and also what they're less well-suited for. That's another good example of a kind of question that people are really bad at answering,
00:03:11 --> 00:03:31

not because they're intentionally deceiving usually, but because we're *not* very good at *anticipating what we might do in the future*, or indeed our *attitudes to future products*, unless you can give somebody a very faithful kind of mock-up
00:03:31 --> 00:03:56

and help them to really imagine the scenario in which they might use it. And then you might get slightly more reliable information. But that's not information I would ever really rely on, which is why *anticipating future product design is such a challenge* and interviewing isn't the best way of getting that information.

When to Conduct User Interviews

User interviews are used both on their own and in combination with other qualitative and quantitative research methods. So, knowing how to conduct interviews will also help you with many other types of research. It also means that how you use interviews in user research and for what purpose varies widely. Let’s look at some of the most common uses for interviews in user research.

For Exploration

User interviews are often conducted during the exploration phase of a design project, before a clear concept has been defined or before a major redesign. You can perform user interviews at the beginning of a project in order to obtain a better understanding of your potential users and various aspects of their everyday lives that are of interest to your project. Context is usually important in order to understand different use cases, because it enables people to “show” what they mean and because it moves an interview to the users’ domain. Consequently, these types of user interviews are often performed in the context where a concept is intended to be used (e.g., in the user’s home, place of work, etc.). When done in the exploration phase of a project, user interviews can form the knowledge basis for personas, scenarios and the like — depending on how you choose to analyze and report on your research. User interviews can also form the knowledge basis for further user research — e.g., to find out what questions are most relevant in a larger survey study.

Women holding ceramic bowls. — User Interviews are often used during the exploration phase of a project in order to explore unknown use contexts and use cases.

© Gustavo Fring and Pexels, CC BY-SA 3.0

In Combination with Usability Testing and Formal Experiments

In controlled experiments and usability tests, you will often measure predefined quantitative criteria such as how many errors the user performs or how long it takes to complete a task. It’s also common to evaluate the user’s experience using questionnaires with a rating scale — e.g., “Please rate how easy it was to locate feature X on a scale from 1-5.” When you use objective criteria and questionnaire ratings, you’ll find them to be straightforward ways to collate your data, allowing you to perform different statistical analyses. However, predetermined measures don’t allow for more exploratory analyses of the user’s experience; so, formal experiments or usability tests are sometimes finished with a follow-up interview.

Here, it’s common to ask the user questions that make it possible to understand the reason behind the user’s actions and experience. The users can explain why they liked or disliked certain features, and you can also ask them to explain why they performed unexpected actions. User interviews are mostly used during studies involving a limited number of participants, such as classic usability testing, and more rarely in large-scale experiments involving a lot of participants.

Illustration depicting the benefits of qualitative insights. Primarily about attitudes and behaviors. Important in user and social research. Typically done with relatively small numbers of participants (often 10 or fewer). Useful source of ideas and explanations. Often a precursor to quantitative research. — Interviews are sometimes used in combination with usability tests and formal experiments in order to gain qualitative insights into the user’s experience.

Concept Explorations

Another common use for user interviews is to explore concept ideas during the early stages of the development process. One way to do this is to show users illustrations or early models of the concept idea so as to obtain their feedback. Concept exploration interviews provide valuable input, but they can be difficult to perform without influencing the participant. Participants are normally eager to please and might be overly positive about your ideas if you are not careful. You should stick to asking users about the problems and desires they have in relation to your problem space, but try to avoid asking them to evaluate specific solutions.

Another way to conduct interviews for concept exploration is to perform interviews in which you ask about the areas relevant to your concept idea, without revealing the nature of your concept. For example, let’s say you are working on a home banking system and you have some ideas as to how to help your users get a better overview of their monthly budget. Rather than present the idea to your users, you would interview them to explore how/if they currently budget and what options they would like to see in the future. Conducting concept interviews in this way will allow you to collect valuable information to inform your concept without asking the participants directly what they think about your ideas. You can also combine the two methods, by presenting your ideas during the last part of the interview.

This video is a great example of how Google Ventures combine user interviews, usability tests and concept validations.

In Combination with Observations

Observations of the users’ actions and context are often combined with interviews. The advantage of conducting interviews in context is that users can show you how they do something. It can be difficult for users to tell you how they use products or perform everyday activities, because they may not remember or they don’t know. If you are interested in how users do something, you are better off watching what they do and then interviewing them about anything that was unclear. There are different ways of doing this. In contextual inquiry, interview and observation are completely integrated. You ask the participant questions at the same time as you observe them interact with a product or perform a daily activity. The user is also asked to explain their interaction as though explaining it to a novice user. In other situations, you do the observation first and refrain from asking the participant any questions until afterwards — to avoid influencing their behavior. If you are performing video observations, you can also get great insights from showing the recordings to your participants and interviewing them about it.

The Take Away

When done correctly, user interviews can provide valuable insights into what the world looks like from your users’ perspectives, but you need to know how to use them and when to use them. It’s important to realize that an interview is different from a normal conversation and you should be careful not to ask your participants questions that they don’t know the answer to. Interviews can be used on their own and in combination with many other types of user research, making them one of the most widely used qualitative user research methods.

References & Where to Learn More

See Ann Blandford’s encyclopedia chapter on Semi-Structured Qualitative Studies.

Jakob Nielsen explains what user interviews can and cannot do in this article.

“How to do a user interview (from Google Ventures updated)”, Konstantinos Kazakos, 30 March 2016

For a detailed explanation of qualitative research methods, we recommend these books:

Qualitative HCI Research: Going Behind the Scenes, by Ann Blandford, Dominic Furniss and Stephann Makri, Morgan & Claypool Publishers, 2016

Interviewing Users. How to Uncover Compelling Insights, by Steve Portigal, Rosenfeld Media, 2013

Pros and Cons of Conducting User Interviews

What Are User Interviews?

Pros and Cons of User Interviews