11. Chapter 11: Curated List of Research techniques

Since this book is intended for both designers and non-designers, we offer you a brief description of a curated list of research techniques. Our objective is not to turn our readers into expert user researchers, since we strongly recommend including a professional user experience researcher as part of the enterprise gamification team. We introduce a curated list of user research and market research techniques pertinent to gamification in an enterprise context. Each of these techniques could be used individually or in combination, based on your objectives.

We also refer you to additional references to go beyond the basics.

11.1 Observation

This technique focuses on seeing what the users actually do as opposed to what they say they do.

11.1.1 Site visit / Field research

This refers to research conducted outside a traditional lab setting, in a user's natural work environment. It involves visiting the site where the product is used and observing the usage in action. It can reveal interesting insights on environmental circumstances affecting the usage of the product, and supplementary tools and work-arounds used along with the product.

For more information, refer to:

Courage, C & Baxter, B. 2005. Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques.

11.1.2 Contextual inquiry

The contextual inquiry research technique combines observation with interview-style question and response. Participants get to explain their actions or "think aloud" as they work through a task or activity.

For more information refer to:

Holtzblatt, K.Wendell, J.B. & Wood. S. 2004. Rapid Contextual Design: A how-to guide to key techniques for user-centered design.

11.2 Surveys / Questionnaires

Surveys or questionnaires are useful to gather information on the profile of the user, his or her job responsibilities and opinion of the current version of product (if available) or similar product (if this is a new release). It is easy to collect both quantitative and qualitative information using surveys. Surveys may be online or face-to-face. Online surveys may be conducted using tools such as SurveyMonkey (footnote 1). Face-to-face surveys may be conducted in combination with observational techniques such as site visits or in a usability lab.

It is important to know how to ask the right questions the right way to get quality input for design. This means not asking leading or confusing questions.

For more information refer to:

Alreck, P.L & Seatle, R.B. 1995. The survey research Handbook

Salant, P. & Dillman, D 1994. How to Conduct your Own Survey

11.3 Focus Groups

A focus group is a qualitative research technique where a group of individuals are asked their opinions, perceptions, beliefs, attitudes or practices regarding a product, service or concept. It is important to pay attention to group dynamics when conducting a focus group since the loudest voice may dominate the conversation and drown out other opinions.

For more information refer to:

David W. Stewart, Prem N. Shamdasan Focus Groups: Theory and Practice

11.4 Interviews

Interviews are a "guided conversation where one person seeks information from the other." An interview may be conducted in conjunction with other requirements-gathering activity such as a site visit, or as a solo activity.

There are various types of interview you can choose from based on your project needs and constraints. Interviews may be conducted remotely (via the phone), or face to face. A structured interview is one where the list of questions is prepared in advance and the researcher tries to solicit answers from all participants. A non-directed interview is one where the interviewer primarlity listens to the subject and provides minimal input or direction.

For more information refer to:

Weiss, R.S. 1995. Learning from Strangers: The Art and Method of Qualitative Interview Studies

11.5 Diary studies

A diary study involves asking the test participants to record and report their experiences related to a particular subject over a period of time. Depending on the type of study, participants may use paper diaries, emails, twitter or a combination. Such studies can be flexible and easy to execute. They are particularly appropriate for understanding mobile device usage since it allows the user to provide their input on-the-go.

Like most research methods, diary studies need to be well designed and have a focus to be effective. A poorly designed study may yield a lot of data that may be difficult to sift through to create meaning.

For more information refer to:

Sharon, T. 2012. It's Our Research: Getting Stakeholder Buy-in for User Experience Research Projects

11.6 Brainstorming

Brainstorming is a tool for creative problem solving, wherein a group of people come together to contribute ideas spontaneously. It is particularly useful when you want to break out of stale, established patterns of thinking, so that you can develop new ways of looking at things. When a interdisciplinary product team brainstorms to come to a common vision of the solution, it helps get buy-in for the chosen solution.

For more information refer to:

Kelley, T. 2001. The Art of Innovation: Lessons in Creativity from IDEO, America's Leading Design Firm.

11.7 Gamestorming

Gamestorming, as the name suggests, refers to the use of games for brainstorming. The term Innovation Games also refers to this technique. Presenting the problem in a game format suspends some of the normal protocols of life and frees the participants to think creatively to solve problems. For example, if the goal is to prioritize a list of features in a product, gamestorming may involve giving each participants a limited set of resources and allowing them to buy / bet on features to see which ones come out on top.

For more information refer to:

Gray, D. 2010. Gamestorming: A Playbook for Innovators, Rulebreakers, and Changemakers.

11.8 Web Analytics

Web analytics refers to gathering and analyzing usage data to gain insights into consumer actions and attitudes. Tools such as Google Web Analytics and Omniture have made it possible for companies to adopt a real data driven approach to understanding usage patterns to optimize the experience for the user. In the case of gamification, it is very useful to know the impact on player behavior to adjust and optimize the strategy as needed.

For more information refer to:

Kaushik, A. 2009. Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity.

11.9 Playtesting

A playtest is a type of usability testing, in which a game designer tests a new game for bugs and design flaws before release. The target player types are recruited via various methods, and are given the game to play. The designers observe the participants and study usage statistics to collect qualitative and quantitative data on the product. They then iterate to make the product better. This practice is beneficial to gamification as well.

For more information refer to:

Schell, J. 2008. The Art of Game Design: A book of lenses.

11.10 A/B Testing

A/B testing is an experimental approach to user experience design. It presents two versions of a website (Option A and Option B) to the user, and analyzes users' behavior. Typically, it tries to track the effect of the differences of the two options against a desired goal. For example, if a website is trying to increase click through rate, they may present a version to one set of online users, and a different version to another. They could analyze if these differences have any impact on the metric they care about.

For more information refer to:

McFarland,C. 2012. Experiment!: Website conversion rate optimization with A/B and multivariate testing.

11.11 Other research methods

When designing enterprise products, it is helpful to know the domain via online research. Researching competitors is an important part of the initial 360- degree research for any product. Analyst and market research reports usually provide good insight into industry trends and business practices.

Professional communities like LinkedIn offer groups for specialized categories of users. Browsing such communities can provide a way to build empathy for your target users by getting a glimpse of their view of the world.

11.12 Customer Co-Innovation Partnerships

Enterprise software companies creating standard applications for business practices need intimate knowledge of work practices in their customer's organization. For established business processes, this information is obtained through recruiting target users that fit the user profile into a user research project. When it comes to innovative business processes, however, the normal recruiting methods fall short due to:

• Need for Customer Trust: True breakthrough innovation such as gamification in the enterprise software context, requires an in-depth understanding of the domain, business process, and employee motivations. Customers are hesitant to share these details with a standard software maker since this information could be shared with their competitors.
• Limited supply of target participants: The type of player may not yet widely exist in the population and therefore the availability is limited. For example, if we are building a solution to leverage social media in call centers, this business practice is still emerging and therefore the available pool of recruits is limited. This increases the time needed for recruiting.
• Innovation takes iteration: When creating a transformational product, the team needs to be prepared to try a few different options. This means even more rapid recruiting to facilitate rapid iteration and feedback cycles. This, along with the limited supply of participants to begin with makes research a challenge for innovative products, and gamification is no exception.

Customer Co-innovation Partnerships addresses these issues by:

• Upfront agreement building trust: By its very nature, this is a partnership agreement between the customers and software makers to innovate. Non-Disclosure agreements are signed by both parties, to assure each other that the information gathered will be used to inform the design activities, and any sharing beyond this will be strictly restricted. Similarly, the customers will have access to early designs and prototypes, and such artifacts may not be distributed. This builds trust on both sides, and enables sharing of insights.
• Target player identification: As part of the upfront co-innovation partnership, the customer can help identify the right people in the organization who can provide input. Since they know their employees better than the external software vendor does, it reduces recruitment effort and increases the quality of the interviews and feedback.
• Iterative validations: Once the correct target participants have been identified, reaching out to them for periodic validation and input is usually not an issue. Studies may be shorter since the researcher need not pack all the research questions into one study.
• Innovative research methods: Once the trust is established via the co-innovation agreement, the team may explore innovative research methods such as Gamestorming to generate many ideas for innovation in partnership with the customer.

There are benefits for the customer too:

• Early influence: Customers who participate in co-innovation have early influence over the product. They are shaping the product to best fit their business process vision.
• Early adoption: These co-innovation partners get the right to be early adopters before their competitors. This head-start advantage could translate into an opportunity to increase market share away from their competition.
• Cost savings: Since the product was designed to meet their needs, the implementation, training, and adoption costs are reduced for the customers participating in co-innovation.

We recommend having three to six co-innovation customers.Fewer than three runs the risk of becoming a customer development project for these specific customers and more than six is hard to manage due to administrative logistics.