If a randomized controlled trial is unfeasible, quasi-experimental designs can to some extent be used as an alternative way to estimate the causal effect of an intervention. Quasi-experimental designs involve the estimation of the counterfactual, that is, comparing the observed results to those you would expect if the intervention had not been implemented, from a comparison group that has not been created at random. Quasi-experimental designs are less robust than randomized controlled trials, but are often more practical in application, particularly in retrospective studies or in prospective studies when randomization is not possible. Quasi-experimental designs can be of two types. In the case of large samples, and even more so with big data, but randomization is not possible, quasi-experimental designs typically include sophisticated analyses of data, such as: One, Difference-in-differences, comparing the before-and-after difference for the group receiving the intervention, where they have not been randomly assigned, to the before-after difference for those who did not. Two, instrumental variables estimating the causal effect of an innovation by identifying instrumental variables, that is variables that impact on outcomes by affecting a key independent variable. This option can also be used to control for measurement errors. Three, matching. Statistically creating comparable groups based on an analysis of the characteristics of the participants included in the groups. Four, regression discontinuity. Assigning a cutoff, or threshold, above or below which an intervention is assigned and then comparing the outcomes of individuals or other units, such as firms, just below the cutoff point with those just above it. All these estimation methods, which will be further explained in the next modules, allow to approximate an experimental design and identify cause-effect relationships. In the case of small samples, multiple matched or matched pair case studies, provided that all other things are as equal as possible, might offer some information about the effect of the introduction of a certain innovation. Consider, for example, the case of a company that wishes to invest in new digital technologies to improve plant logistics. It might think of adopting automated guided vehicles to improve speed and accuracy of material flows to feed the production lines. Automated guided vehicles are portable robots that follow markers or wires in the floor, or uses vision magnets or lasers for navigation. In industry 4.0 environments, they are integrated with other production technologies through machine-to-machine communication technologies, so that all the information related to production scheduling is digitized. The company is considering investing in automatic guided vehicles, but would like to have better information about the speed, accuracy, and cost improvements associated with the adoption of such technology. The company has four plants and each plant has three production lines. What should the managers do? They could run a quasi-experiment. First, record speed, accuracy, and cost data for the production lines before the intervention. Second, pick one or two matched pair production lines, so that each pair is as similar as possible on the largest possible number of meaningful covariates. Third, pick randomly one production line in each pair and use that as the experimental line in which to introduce the automated guided vehicles, keeping the other production line unchanged as a control. Fourth, implement and bring automated guided vehicles to regime and standard operations in the experimental production lines. Fifth, measure speed, accuracy, and cost before, during, and after the experiment, checking if there is a significant improvement in the experimental lines vis-a-vis the control lines. If this is agreed upon with the technology supplier, so that for example the company does not commit to buy the new technology before the test, this is an effective way to get better information and make a more informed decision about the adoption of a new technology. Summarizing, randomized controlled trials and quasi-experiments that mimic the conditions of randomized controlled trials when these are not applicable, are effective ways to get, often at negligible cost, better information, make better estimates, and reduce the probability of incurring in innovation decision mistakes. Randomized controlled trials and quasi-experiments can be of different types, and can be conducted in the field or in laboratories, simulating the field conditions through technologies. The specific type should be chosen contingent upon the specific company need and business context.
