We are now ready to discuss the logic of the key decision rule of our analysis: V̂ greater or equal than V*. Consider a table like the one we discussed in the previous session, where we look at the joint probabilities of two random variables. The prediction V̂ and the true value of investment V. V̂ is a random variable because under different conditions decision makers make different predictions. For example, because they use different samples. Each predicted value they can make is then associated with some probability. To illustrate the logic of the decision rule, we developed four numerical examples in which V̂ and V can take values -2, -1, 1 and 2. The first example is the benchmark. Consider this table: the probabilities in the cells are the joint probabilities of the realization of V̂ and V, respectively, in the row and in the column. For example, the probability that V = -1 and V̂ = 1 is 10%. The probabilities in the last column are the marginal probabilities of V̂. That is, the probabilities that V̂ = -2, -1, 1 or 2, irrespective of the value of V. The probabilities in the last row are the marginal probabilities of V. The probabilities of V̂ tend to be higher around the same value of V. For example, the probability that V̂ = -2 is 10% when V = -2, and declines to 5%, 5%, and 0% when V = respectively -1, 1, or 2. Similarly, the probability of V̂ = 1 is higher when V = 1, and so on. Good estimates are more likely to exhibit values closer to the true value. To obtain the optimal threshold, suppose our decision-maker has estimated V̂ = -2. She then knows that the expected value of V, conditional upon V̂ = -2, is equal to the sum of each realization of V, multiplied by the probability of that realization, conditional on V̂ = -2. This is equal to -2 times 50%, -1 times 25%, + 1 times 25%, + 2 times 0%, which is equal to -1. 50%, 25%, 25% and 0% are the probabilities in the cells, divided by the marginal probability that V̂ = -2. They are the probabilities of each realization of V, conditional on V̂ = -2. Suppose, instead, she estimated V̂ = -1. We compute the expected V, conditional upon V̂ = -1, in the same way. It is equal to -2 times 1/6, - 1 times 50%, + 1 times 33%, + 2 times 0%, which is equal to -0.5. Again,1/6, 50%, 33%, and 0% are the probabilities in the cells divided by the marginal probability that V̂ = -1. They are the probabilities of each realization of V, conditional on V̂ = -1. Compute in the same way the expected values of V conditional on V̂ = 1 or 2, which are equal to 0.5 and 1. It is easy to see that whenever our decision maker estimates V̂ = -2 or -1, the expected values of V conditional on the signal are negative. Conversely, when she estimates V̂ = 1 or 2, the expected values of V, conditional on the signal, are positive. The optimal threshold is V* = 1. This is because when V̂ = -1 or -2, the expected value of the action is negative, while it is positive when V̂ = 1 or 2. Thus, the investment is worth only when V̂ is greater or equal to 1, which is the optimal value of V*. If the decision maker said V* = -1 or -2, she would act whenever V̂ is greater or equal to this value. But this means that she would make the investment even if V̂ = -1 or -2, which produces negative returns in expectation. If she said V* = 2, she will not act when V̂ = 1. But when V̂ = 1, the expected value of the investment is positive. Therefore, V* = 1 is the optimal threshold, because the decision maker will never make negative expected returns, and will never miss a positive expected return opportunity. The probability of type I error is the probability that V̂ is greater or equal to V*, conditional upon V smaller than 0. That is, with V* = 1, V̂ = -1 or -2. This is the case in which V̂ is greater or equal than V*, which encourages the decision maker to act, but the true value is actually negative. In our table, this probability is equal to the sum of the probabilities in the four cells with V̂ = 1 or 2, and V = -1 or -2, divided by the sum of the marginal probabilities of V = -1 or -2. We obtain 30%. The probability of type II errors is the probability that V̂ is smaller than V*, conditional upon V greater or equal to 0. Now you sum the probabilities in the four cells with V̂ = -1 or -2, and V = 1 or 2, and divide by the sum of the marginal probabilities of V = 1 or 2. This probability is equal to 30%. As an exercise, compute the probabilities of type I and type II errors, if the threshold was V* = -1. You will obtain 70% and 10%. A lower threshold, then, implies a higher probability of type I error, and a lower probability of type II error. That is: When you act, you make more mistakes, false positives, but miss fewer opportunities, false negatives. We have seen that the optimum threshold is one, that's the balance of type I and type II errors is such that the threshold V* = 1 produces greater value. We now consider a second numerical example in which there are higher opportunities than in the benchmark. There could be more innovation opportunities or higher demand. We represent this by setting marginal probabilities of V higher than in the benchmark, while leaving the marginals of V̂ equal to the benchmark. As an exercise, go through the same logic of the benchmark and compute the expected values of V, conditional upon each one of the four estimates V̂ = -2, -1, 1 and 2. You will see that the optimal threshold is V* = -1. This means that, now, even if we observe V̂ = -1, the expected value, conditional upon V̂ = -1, is positive. Why? What is the intuition? The intuition is that you know that you are in a world with higher opportunities. Therefore, even if you make a - quote unquote - “bad random draw of the hat”, that is V̂ = -1, the true value of V is likely to be positive. Of course, there is still information in V̂, in that if you pick V̂ = -2, you do not act. However, your information that the distribution of the exhibits - higher weight on higher values - suggests that even if you obtain a lower estimate than in the benchmark, you expect a positive value from the investment. As an exercise, compute the probabilities of type I and type II errors when the threshold is V* = -1 or 1. This second example also helps to understand the logic of the distribution of V. The probability distribution of V, before observing the signal V̂, is what Bayesian statisticians call the prior distribution of V. The signal changes the probabilities of V, and the probability of V conditional upon observing a specific V̂, -2, -1, 1 and 2, is called the posterior distribution of V. The posterior is affected by both the signal and the prior. Which is why a prior suggesting higher business opportunities induces a lower optimal threshold. In our third example, we consider the same marginals as in the benchmark. But now we have higher marginals for lower values of V̂. This corresponds to the case of a pessimist decision maker. For equal priors of V, she makes more conservative estimates. Show us an exercise that in this case you obtain an optimal threshold = -1. What is the intuition, again? If the decision maker is a pessimist, whenever she estimates a low V̂, it may still correspond to a relatively high probability of V. There is information in her estimate, in particular: V̂ = -2 suggests a negative conditional expectation of V. However, we take into account her pessimism, because even if she states V̂ = -1 the conditional expectation of V is positive. The takeaway of these examples is that they inform what we could do in practice. If our experience, background, or intuition, which represent our prior, tell us we live in a world of opportunities, we probably want to set a more lenient threshold, because even if we obtain low estimates, there may still be value in acting. Similarly, if we deal with a pessimist decision maker, we want to lower our threshold, and the opposite is true with an optimist decision maker. By maneuvering the threshold we bring back our experience, background, and intuition in the decision. This also begs the question: how a pessimist or overconfident decision maker can correct themselves? As we will see in the next session, decisions are rarely taken by individuals. There can be a different decision maker than the individual who makes the prediction, for example a boss. The boss can mitigate her pessimism or optimism. In fact, most often decisions are collegial, and the discussion mitigates pessimism, optimism or takes into account priors about opportunities. In these discussions, whoever believes others are optimist, or pessimist, or there are high or low opportunities, could propose mitigations by maneuvering the threshold. The discussion will then mediate across these priors. Finally, we discuss the implications of a scientific approach. As discussed in the previous sessions, problem definition, theories, and tests reduce biases and logical fallacies, and provide better interpretations of signals. This reduces the probability of type I and type II errors. We represent this in our table by keeping the same marginal probabilities of V̂ and V like in the benchmark. And by increasing the probabilities that the prediction V̂ takes values close to the corresponding V. Specifically, we increase the probabilities that V̂ = -2 when V = -2, and similarly for -1 and -1, 1 and 1, and 2 and 2. As an exercise, show that the optimal threshold is again V* = 1, like in the benchmark. You will also show that the expected value, conditional upon taking action, that is conditional upon V̂ = 1 or 2, is higher than in the benchmark. This is the outcome of the scientific approach. By increasing the probabilities that you estimate a value close to the true value, or - which is the same - by reducing type I and type II errors, you obtain a higher expected value, conditional upon the optimal decision rule.