Why and When to Use Surveys

Looking at when to use surveys relative to  the product or service lifecycle? Well, you might have an existing solution and for that you  may well want to consider a survey after every *major release*; or perhaps on a *calendar basis*, every quarter you might run a customer satisfaction or a user satisfaction survey. And that allows you to keep a pulse on things to know how your offerings are faring

in your users' or customers' eyes. If you have specific ideas for improvements, you can also ask users about that, ask customers to tell you the sorts of things your competitors are doing that they like or perhaps ask them about *sticking points* in your current solutions: what is it that they find problematic, where would they like to see improvements? And, of course, in those kinds of areas, you are talking more about open-ended. But certainly, if you've got a list of things that your  competitors are doing, it's very easy to ask people

whether they would be interested or would find  those particular features useful. For new solutions, you often end up using quantitative research, which is what a survey is, to what we call "triangulate" – get extra data about – back up – qualitative research that you've done. So, you might have gone out and done some contextual inquiry and you might have some really exciting ideas about new product directions, and you want to make sure that that makes sense for the majority of your customer base.

You wouldn't just go out on the strength of a dozen interviews and launch a new product or major revisions to a product  or service. So, the use of a survey is really almost essential in those kinds of cases. *Alternatives* – I mentioned contextual inquiry. The great thing about contextual inquiry is that it's grounded. We go out and speak to real people about real situations in a fairly – what's called – *ethnographic way*.

So, we're trying to do it in their own settings, where they  would be using this product or service. And a contextual inquiry is *extremely exploratory*. So, if you start hearing about certain ideas on a regular basis, you can start asking *more* about that and try to expand the scope of your inquiries to cover these new concepts and find out a lot more about the product or service that you should be providing, as opposed to the one that you perhaps currently are or were planning to.

*Semi-structured interviews* – well, these are a really  important part of most qualitative research and,  in fact, is used in contextual inquiry as well, but  they aren't necessarily as well grounded. We don't necessarily go out into the user's environment  to do those, but one of the attractions there is that we can start off in both of these examples – contextual inquiry and semi-structured interviews – start off with a collection of initial questions and then explore from those.

So, we might have only a short list of topics that we definitely wanted to cover and we'll let the conversation ramble into interesting connected areas – not just ramble in general, by the way; "interesting connected areas" is an important part of that. You want to make sure that you're still within the focus of your inquiry, of your research. *Card sorting* – it's really good for early research for finding the relationships between concepts. We've got concepts on cards, and we ask people to sort those cards

into groups, either of their own creation, so they're allowed to make the groups up themselves – that's called an *open sort* – or a *closed sort*, where we provide the groups and we want to see if people agree with where they're putting things; and in between, of course, those two is something that I call a *hybrid sort*. It has different names. And there are other early-testing tools, which we do talk about elsewhere. Those are, I should say, *tree sorting* or *tree testing* and *first-click testing*, where we're trying out very specific things; we  give users a goal, and we try to see how they

address that goal with the solutions that we're thinking of providing. So, in the case of tree sorting, it's actually *menu testing*. So, the tree is the menu, and we say, "Where would you find this?" / "How would you do this on this site?" and you show them  the menus a step at a time. And there is no site yet. There's just a listing of the menu items  in a step-by-step progression. So, they're shown the top-level menus, they're shown the second-level menus, etc., as they navigate through.

So, it's really easy to do and you get some really good hard data  out of that. And, similarly, with first-click testing, you might have just wireframes or really early prototypes; it can even be sketchings and you ask people to try to achieve a goal with these designs. You record where they click and how they try to achieve that. So, it's actually first-click testing is the most interesting part of that: Where do they focus their attention initially when trying to achieve those goals? So, these are all alternatives to asking people about things.

And, of course, in these latter cases we're talking about seeing people do things  rather than asking them their opinions, which is a much more reliable way of getting data – not that surveys are entirely unreliable; that's not the case – but first-hand information about what people  do rather than what they talk about doing is much safer. And this is a pretty typical field-working experience. The guy on the right has a PDA or phone.

Hopefully, it's a multiple-choice questionnaire  he's asking because it's really very hard to make notes on a device like that, but this is the kind of situation where you can direct the questioning according to how the participant is answering. So, this is an alternative to surveys.