Another interesting example is what Amazon did in its early days to test a key belief about future value creation, or hypothesis as described by Michael Schrage, in his book, ‘The Innovator’s Hypothesis’. Greg Linden from Amazon in the company's early years observed that there were some trends in what existing customers were purchasing. In other words, customers were often buying related products: for example books of the same type, say thrillers. He believed that Amazon could create an even better shopping experience for its customers if its software could make intelligent recommendations on what to buy based on their past purchases. Linden decided to test his hypothesis by quickly creating a test site for a limited stream of customers. An impressive number responded by clicking on the recommendations and adding the items to their Amazon shopping carts. This was a way for Amazon to quickly test if the introduction of a new product could work by collecting primary data. While ideally companies will have access to data from past experiences to test their hypothesis, or they might have the option to collect new data as in the case of Amazon, this is often not the case. When a new business idea or a new product requires a novel data collection effort, companies might want to consider the value and the insights that come from small data. Martin Lindstrom describes how to use these data to test basic assumptions, beliefs or hypotheses in his book titled: ‘Small Data: The Tiny Clues That Uncover Huge Trends.’ The approach presented by Lindstrom combines keen observations of small samples and applied intuition to gain insights into customer behaviors and interests. Lego used this approach when in 2002 their sales were in stable decline. A study of existing data on customers reveal that with an increasing adoption of new technologies the attention span of their customers was decreasing. Since creating objects from tiny blocks took considerable effort, Lego's solution was to introduce bigger blocks that could form objects more quickly. However, this strategy wasn't working, and Lego’s sales kept on declining. This test showed that what data suggested as a general trend, the fact that their customers had a more limited attention span, was not necessarily what was driving the decline in sales of Lego products. A team of legal researchers in an attempt to better understand customers visited the home of an 11-year-old boy in Germany. Among other things, they asked him what he was most proud of in his room. The young kid pointed to his worn out sneakers. His sneakers were the evidence that he was the best skater in town. And Lego's team realized that while data correctly revealed that the attention span of young children was becoming shorter, they would still spend thousands of hours on their passion. Lego revamped its strategy based on these insights, and decided to bring back the smaller blocks. But what happens when founders of a startup want to test key hypotheses about the feasibility of a new business idea? In other words, how can hypotheses be tested when there is no existing data? It is still possible to test key hypotheses to learn what really works. Two different examples come to mind: Dropbox and Food On The Table, two startups that use very different approaches. Dropbox is a personal cloud storage service, sometimes referred to as an online backup service, that is frequently used for file sharing and collaboration. When Drew Houston and his team of engineers started working on Dropbox, it was not clear if customers wanted to use its product. This was a particularly important problem, as they believed that Dropbox was solving a problem most customers didn't know they had. Their key hypothesis was that if this software could work perfectly, customers would flock to it. The key challenge here was that, unlike in the Amazon example, it would have been impossible for Dropbox to build a prototype. Hundreds of hours of coding and testing would've had to go into this product. To test his hypothesis, Drew created a three-minute video to demonstrate how the technology behind Dropbox would have worked. This video drove hundreds of thousands of people to the website and the Beta waiting list went from 5,000 to 75,000 people in the space of a day. In this case, the video was what allowed this company to gather data to test whether their initial assumption was right or not. Food On The Table, instead, went for a completely different approach when dealing with a lack of data to test their hypothesis. Food On The Table was a Texas-based startup that created weekly meal plans and grocery lists based on the food a family typically consumes. Food On The Table helps find the best deals and ingredients and deliver them to customers' homes. The company is no longer operating under its name but it has been acquired by Food Network. In these cases, while funders had to test if their business idea was feasible. Their key hypothesis was that customers were willing to save time by using an online service that provided the convenient service of groceries delivery. Even in this case, the key challenge was building a prototype. For their website to work, they would have needed an algorithm that match recipes with ingredients on sale at local grocery stores. So, to test their hypothesis the founders of Food On The Table started by delivering the service manually to one customer who signed up for $9.95 per week. The founders would review what was on sale at the customers’ preferred grocery store, and hand them a prepared packet containing a shopping list and relevant recipes. They would then solicit their feedback and make changes. After a few weeks, another customer signed up. Each new customer got this concierge treatment, which provided tremendous opportunity to test their key hypothesis and learn how to fine-tune details of their offering. At times, some of their hypotheses were not confirmed. However, failed tests of hypotheses turned out to be just as useful as successful ones. The founders engaged in a process to get to the root cause of why their assumptions weren't true. The key message of this video is that hypothesis testing can take place in many ways. Companies can use primary data that they have on their customers’ habits to test hypotheses. However, when dealing with a new product or new feature, there might be the need to collect new data to test key assumptions behind new products. While large samples have significant advantage in allowing us to estimate more precisely our effect of interest, in many situations even small samples can provide important insights and tremendous learning opportunities.
