Paired Interviews – applying pair programming thinking to user research

As I’ve said before, Agile is not a specific methodology or set of practices; it is a way of thinking about creating software products.  If you are a UX designer, once you understand and are able to adopt an Agile mindset, you’ll be able to reshape your practice, such that it takes full advantage of Agile thinking. Manifestations of this include replacing slow-to-create detailed specification documents with fast and light-weight conversation-centered design documents.

Here is another such manifestation, in this case applied to user research, something I call Paired Interviews.

A paired interview session, in which users interview one another

A paired interview session, in which users interview one another

The paired interview method applies the thinking behind pair programming toward user research.  Instead of doing the more traditional one-on-one interviews with end users and domain experts, the researcher instead asks end users to interview one another, while moderating their discussion. The end result is a research findings artifact, commonly in the form of a prioritized list of story cards or a story map, created by the session participants during the session, rather than by the researcher after the session.

Why are paired interviews effective?

Paired interviews can often be a significant time-saver.  They effectively allow a single researcher to interview many users at the same time, by having users double as interviewers. Additionally,  these users will be able to ask questions using appropriate domain-specific terminology, reducing interview churn and the need for clarification.  They are also likely to be able to detect domain-specific subtleties in the interviewees responses and be able to then ask the ever-important follow-up questions.

Another possibly less obvious advantage of this model is its social impact: seeing and hearing others, who are members of the same demographic or the same organization, speak openly about the topic at hand, can encourage them to also want to add their share of insights and ideas for improvement.  Somewhat ironically, the din of noise from these multiple conversations also creates a kind of privacy bubble for each pair, helping them feel more comfortable opening up about possible frustrations with the current product.

What are some drawbacks of this technique?

The most obvious disadvantage is that you are not able to have an in-depth discussion with each user and therefore possibly missing some critical details. However, paired interviews of course do not exclude the ability to do one-on-one interviews.  Close observation of participants during a paired interview session often can be useful in helping to determine who might be worth following up with for a one-on-one interview.

Paired interviews tend to also require more preparation than a one-on-one interview, which can often happen ad-hoc, i.e. with little or no preparation.  As we will see, however, the time invested in preparing and running a paired interview session is likely to be well worth it.

Running a Paired Interview Session

Here is a super-brief overview of running a paired interview session.  Prior to running a session, you should (obviously perhaps) have familiarized yourself with the product domain and the context in which the product being designed is expected to be used.

Selecting Participants

Try to include a broad range of users. This may mean including both advanced users and those new to a possible legacy product. At the same time, you want to be sure that the pairs interviewing one another are able to relate to what the other person is saying.

Number of Participants

This session can be completed with anything from a single pair to a large group of pairs.  Many pairs is better, since participants are likely to feel less self-conscious in interviewing one another. For larger groups, consider having additional team members assist you with moderating the interviews.

Session Intro

Setting discussion boundaries is key.  If you are doing the interviewing, you are in control of that, but here you will be relying on the users themselves for this.  The project goals, which are a good idea to present, set natural outer discussion boundaries, but you can feel free to narrow it even more. For example, a project goal may be to replace an old desktop publishing system with one that is cloud-based.  Ask participants to try to keep their discussion focused on the old system and their ideas for a new one.

And don’t forget to explain the concept of paired interviews, since this is likely to be new to many of those attending.  Explain that the interviews effectively are conversations, in which both parties are interviewing one another at the same time.

Conversation starters

While many users often are eager to discuss things that frustrate them, such as their current crappy enterprise software, others may need some assistance with getting their discussions underway.  Therefore, present a set of conversation starters.  Here are some examples.

  • “Describe a really bad workday you had recently.”
  • “What is your least favorite feature or work activity?”
  • “What is your favorite feature?  What makes it great?”
  • “What would be an ideal workflow?”
  • “What is the most time-consuming part of your day?”

Try to tailor these to be more specific to the project domain or context.

Pairing up

Ask participants to pair up. Allow participants to self-organize, i.e. to determine who they would like to interview and be interviewed by.  Forcing someone to pair up with someone else may backfire, since you may put two people together who do not get along.  If you are left with an odd number of participants, such as due to a no-show, ask that one person to join one of the pairs, and have a three-way conversation.

Capture conversation with cards

Provide participants with markers and sticky notes.  Ask each participant to jot down a few word capturing each key point made by the person they are interviewing.  Some participants may be new to this way of taking notes, so consider providing some examples.

For example, if the person you are interviewing says: “Sometimes I am in the middle of filling out a form and have to go ask someone a question, and then when I come back, there is a message that my session has ended, and all my form data is lost.  Super frustrating!”

…jot down something like: “Session time-out, form data lost!”

The key here is to jot down just enough that it will be possible to trigger a more detailed conversation about the topic at a later point.  Explain that this approach both allows for taking notes in real-time, and allows both the interviewer and interviewee to see what notes are being taken.  That way, if the person being interviewed says something they feel is a key point, but the interviewer does not make a note of it, they can see this, and ask them to add a new sticky note.

The Interview Timebox

Start a 5-minute timebox.  This will usually trigger an immediate flow of conversation and sticky notes.  Try providing pairs that seem to be struggling with some personal attention. Keep a close eye on the cards being produced by each pair.  If there are pairs with few or no cards in front of one person, check with them to see that both parties have had a chance to interview the other.

At the conclusion of the interview timebox, ask for a show of hands of those who feel they were not able to share everything they would like to.  For those pairs who raise their hands, give them another few minutes to finish their discussion, while others get started with developing and organizing research findings.

Interview Summary

Go around the room and ask the interviewer in each group to use the sticky notes to summarize the conversation.  The reason for asking the interviewer and not the interviewee to do the talking is to allow for capturing both the original discussion and the interviewer’s opinion and possible disagreement. Be sure to note anything they are saying that is pertinent but not on the card.

Iterating

At the end of the summary, ask participants if they would like to complete another round.  You are likely to find that there is value in completing 2-3 rounds.  It is better to err on the side of doing too few rounds here, as you will have plenty of raw material to work with even from a single round.  You also want to make sure you leave at least 20 minutes to half an hour for mapping and normalization.

Mapping and Normalization

Ask each pair to group their cards by category and relative importance. Ask pairs who are reasonably far along in organizing their stickies to move them to a whiteboard or wall. (You can use painter’s tape to create silos for each major area.) Observe this activity very closely.  As you see pairs starting to create categories, look for common categories or workflow steps across pairs.

You will most likely see self-organizing behavior here, such as those with stickies containing the same information placed next to or on top of one another.  If there is disagreement regarding the organization, ask those in disagreement to see if they can resolve their differences or, if not, create their own proposed versions.  Because the discussion is revolving around a constrained topic area or domain, you are virtually guaranteed to see similar headings.  If you don’t, this may mean that there is a larger issue, such as an organization where there is a complete lack of structure and everyone is working in an ad-hoc fashion.

You have here created the beginnings of a story map.

Cross-Pollination

Once all stickies have been added, ask participants to switch pairing partners and walk the map in pairs.  Since they are now seeing not only their own sticky notes, but also those created by other pairs, as well as being in a conversation with a different participant, this may elicit additional sticky notes.

The Resulting Research Findings

The resulting artifact will be a normalized, prioritized, and categorized collection of feature cards created by your users.  It usually is a strong indication of the needs, desires, and preference of the larger user community. The findings have many possible uses, such as to serve as the basis for a design studio, for developing a product road map, for validating or adjusting a product vision.

8 thoughts on “Paired Interviews – applying pair programming thinking to user research

  1. Pingback: The UX of User Stories, Part 1 - Anders Ramsay.com

    1. Anders Post author

      Not sure why you would not expect this to offer much quality. Sometimes one’s own peers are the best people to elicit good user feedback. At least that is what I have found, especially when I want to quickly gather data from a large group of users. Also, keep in mind that the chunking and prioritizing activity that follows the interviews also is key to distilling the raw findings down to some true nuggets of information.

Comments are closed.