So, good research is only as good as how it is that you frame the research question and when you're trying to learn about what contexts preferences needs pin points or even the general usability of a digital product or any product or service, you need to understand five questions that start with W and two that start with H. So, the who, what, when, where, why are really critical because that helps you to understand who's using a product, what it is they need, when they're using it, where, why they're using it. The two how's are also really important because they help you to understand more about how it is that people in a greater sense see a particular product in the context of what it is that they're trying to accomplish in their lives. How many of course is very important especially when you're doing research at scale. Because it's important to know how many people you are creating that experience for or how many people you need to test with, so that you get the right number of folks involved in what it is you're trying to accomplish with your research. So, my thought is that research drives creativity because it enables designers and developers to get a sense for what are the things that people feel comfortable with, what are the things that people need, what are the things that they will tolerate, how far can you push a particular approach? The act of research is also creative because you have to figure out how do we answer these questions, where do we answer these questions, and what are the constraints that we have to work around in order to answer those questions? Research is not easy. One of the things that we find is that understanding is sometimes really hard to make. Sometimes it's because you don't have a good question or an ill-defined question. Sometimes it's because the results themselves are actually inconclusive and it means that maybe you picked the wrong methodology. I'll talk a little bit more about that later. I really want to talk about the fact that research is not ever done in a vacuum. Nobody ever goes and does research just for fun. Because what we really want to be able to do is help to improve a product, to create a product, to solve problems for people or to solve problems that exist within a product. We also sometimes seem to take a step back and figure out when do we need to figure out, what the path is that we need to go down in order to understand what it is that we need to do before we dig deeper into our problems. That really is this more meta type question of, "How do we identify the questions that need answers or the problems that we want to solve?" Then finally focusing on those areas of concern and a product really helps you to make improvements obviously. Think about it. When you're starting to structure a research project, what are the things that you need to think about. The first is why are you doing the research? So, as I mentioned earlier, nobody does research just for fun. So, you're really trying to understand what the organizational priorities are to help to understand what it is that you need to rein the discussion so that you're answering the right question at the right time for the teams that you're working with. So, the way to come up with a really good research question is first to identify that, what's driving the reason why you're doing research? So, you would make a list of priorities, so that you know," Hey, we really want to understand how the product's user experience affects the goals of the organization." So, how those two things interrelate. One of the things that you want to do is make sure that you write them down as goals and as questions, prioritize them, and then rewrite them to be very user specific, so that you can get a sense for what are the tasks that we need to have people do so that we have people experience a particular piece of the product that's going to help us to answer those questions or how do we make sure that we ask the questions that are going to drive people to give us the answers that we need. So, a little bit more about how to do that. The first thing that you need to do is really think about things as a particular problem that you need to solve. So, one might be the number of registrations that we would like to have. People actually complete on the site is lower than what it is that we would like to see. Well, people should be able to register online or registered easily online. So, there are few things that users would need to do to be able to follow through on an online registration. First is they'd need to register to use the site. They need to be able to potentially make online payments and then finally, they need to learn the status of their application because registration may not be guaranteed. The next thing that you need to do is rewrite goals as questions that need to be answered. So, are people actually unable to complete registration? So, are there roadblocks that prevent people from completing the registration process online? There might be other questions that you would want to understand more, rather have less to do with the usability of the site but more about how people feel about the types of information they're being asked to provide. So, for example, do people prefer to register using less personal information might be a fact that you would want to learn more about. Then finally, you'll want to understand more about, who is registering and what types of information must they have so that they are prepared to register when they go online. So, one of the big challenges that a lot of foreign designers come across is especially if they are creating form set enable people to do very complex things, is do they need to use all the information that is being collected? Obviously, you want to understand who your users are and there's more than one way to accomplish this. So, you want to learn their contacts, preferences, needs, and expectations. You can do that by actually doing remote testing through interviews. So, these can be both moderated and unmoderated interviews. As well, you can use surveys. You've heard about that in Cliff Lampe B's class. Then finally, behavioral analytics and we'll talk a little bit more about that later on in this course. The next thing that you want to consider is, what are people doing to solve their problems and what are the tools that they're using to do that? Again, this is something that you can do remotely, as well through both moderated and unmoderated testing. So, one way to understand more about what people are doing to solve their problems is to undertake task analysis by actually having them complete an exploratory or mission-based task. Essentially, what that means is you've set them a task and you don't tell them exactly how to accomplish it but rather accompany them so that you understand what they're doing, what tools they're using, and what is in their environment while they are accomplishing that task. When is an important question because how it is that people engage with products differs depending on what time of day they're using them. So, looking into information that takes into trends around why it is that people are engaging with products at certain times of the day or certain parts of the week, helps you to understand what are the things that we can do to ensure that that experience makes sense for them. So, ways to accomplish that includes surveys, clickstream analytics, and interviews because the clickstream analytics gives you a sense of what people are doing when, where are they, what are the periods of high volume usage on the site versus what are the time periods when people are not using the experience as frequently. Then you can also dig into why are people using those experiences when they're using them and then you can understand how you can make changes to optimize that experience. Where people are using products is also critical. So, that helps you to understand whether people are using their mobile phones in your in-store location versus in their homes in front of their televisions, and what types of information are going to be most helpful to them in those different contexts so that they can accomplish their goals. So, for example, in the in-store experience, you might want to have functionality that enables people to find products on the shelf. Whereas, if people are using a digital product to, for example, tweet along with a television show that they're watching, you would want to provide different sets of functionality. So, here again, diary or longitudinal testing ends up being very important because it helps you to understand where people are using products in the context of what it is that they're trying to accomplish across time, and then also remote testing allows you to get outside the lab and understand how it is that people are using products. Why is obviously the most important question of all, especially for people who are into qualitative research. So, discovering why people have the needs, expectations and behaviors that they do in their own context helps you to make better choices about how you want to enable them to accomplish their goals. So, one of the things that you can do here is to test concepts with them to get a sense of, does this work, do people get it. Then enabling them to try those concepts using a remote tests is even better because that helps you to understand whether or not that approach is really workable for them. How people are actually using a product helps you to understand what gaps currently exists, and understand potentially what types of products you might build that would suit their needs most appropriately. So, field ethnography is especially in the digital contexts becoming a more and more popular mode of enabling researchers to understand how people are currently interacting with the products that they have in their environment, so that researchers and designers can figure out what types of products might be most appropriate to add to that particular setting. How many is really important especially for people who are into the analytics and data side of the equation, because it's important to understand what the scale is of the behaviors observed because you want to understand is it exceptional behavior, or is it normal behavior, or is even behavior that you would expect of a certain segment of users because you really want to get a sense of if we make the change here, how many people will it touch and isn't going to change the behaviors of a significant proportion of the people that we care about most. So, here's where A/B testing and Clickstream analytics are crucial because that really helps you to understand where people are going, how many people a particular change might impact, and as well in a very real way which version works better? So, one best practice that you must always keep in mind is that you should start with your report first. That is because in your mind's eye if you have a sense for what that data is going to need to be, you'll have a sense for exactly what it is that you will need to collect so that you can have findings that resonate appropriately for your audience. So, if you have a sense for what your report is going to look like, that will drive what types of participants you select and then also what methodologies you select. Researchers in every context always pilot a test before they run it with users and that is because inevitably, there will be some problem that you will run across that you may not have anticipated. People may not understand one of the questions that you have in a test or you may have forgotten to add a step or an instruction to enable people to get through the test. So, a few thoughts, it's really important to make sure that you're helping to build narratives. People respond best to stories. So, UX research really helps you to understand how is it that we can build a story that supports the choices that we need to make in order to optimize the user experience. A few thoughts around when you do observations. So, when you're running a test, it actually makes sense, and I'll show you in a moment what this looks like to start in the middle of the week because as I said research is creative, it's very difficult to come up with good answers and insights in moment's notice. Just as a designer requires a little bit of time to come up with a good design approach, a researcher also needs time to figure out what does it all mean. So, give yourself that gift of times that you can come up with the best answer and had that aha moment that helps you to thrive great research and great insights. Also keep in mind that all the studies that you run may not get you the answer that you want the first time around. It may be that the first test that you run gives you inconclusive answers but it also tells you that wasn't the right method, or it also tells you maybe we need to pick a different type of participant and that will help us to get the answer that we need. So, give your self the ability to wander at least a little bit. Pilot testing will also enable you to do this a bit. Finally, practice makes perfect. The more you do user and usability testing, the better you will get at it. So, that thought around doing testing during the middle of the week. One of the things that we find, and this is a very much a practitioner-driven best practice, is that research that starts on a Thursday or Friday gives you that downtime over the weekend to digest what it is that you've seen. So, on that Thursday or Friday and here I'm showing the example of a Thursday, you want to run that test or capture those observations and take your notes. Hopefully, you're doing this alongside your colleagues that you can have some discussion about what it is that you're seeing. On that next day, you can write up your preliminary findings and start talking about what are the things that we can do to make changes or improvements. Over the weekend, relax, obviously it's the weekend. Then the following week what you really want to do is make sure that you're sharing those findings and mailing them down so that people know what the next steps need to be. Now, in this case, you're really a assuming that you are a researcher who needs to write a report, and so that gives you a few business days to do that in a way that makes sense for you. What we find these days is that in an Agile environment, reports are much less important than findings. So, these longer term tests tend to happen less frequently. So, more frequently what we find is that tests are run primarily in the middle of the weeks and sprints run usually like a Tuesday to Wednesday or Wednesday to Thursday cadence, and those tests happen same day and very iteratively and with very small groups of users. So, two very different approaches to how it is that you would potentially do user testing.
