Okay, so data integration and accuracy of the data and also the complexity of the marketing environment itself, presents a lot of challenges in measuring marketing ROI. So for somebody who is trying to start this in their firm or is just getting into this, what advice would you have for someone starting to measure the return on their marketing investments? Well, I think I would start, we've already talked about how complex it can be, but I would start with the basics and not go with the complex stuff until the basics are well taken cared of. By that I mean, the first thing that needs to be done is, if an organization is going to talk about marketing ROI and evaluate their marketing against it, they need to clearly and consistently define ROI. It sounds like a simple thing, but across organizations and even within an organization, people use the term loosely. Sometimes you'll see an article about some study has shown that ROI of something is really good and then you'll read the article and they'll be using the term ROI just to talk about an increase in the number of clicks or something like that. So that's one extreme, but even when people are pretty serious about measuring things, you've got to be careful how the term is used. So for example, often when people use ROI, the most common thing they're referring to is a profit ROI. So it's how much bigger sales times the profit margin that we're generating by the marketing. So how much incremental profit was generated divided by the cost of the market. And when you do that, you get a number like a dollar, maybe break even. Another very common measure is called ROAS. Internal ad spent, or reach our level revenue ROI. It's the same calculation, but instead of applying the profit margin to the final incremental sales, we simply apply the retail revenue dollars to the final equipment sales. So the relative numbers across marketing elements should still make the same decisions, but you can't compare an ROAS to an ROI. And often people start getting ways, again, referring to an ROAS as an ROI. And that's where a lot of confusion can occur. And that's just one example of the confusion. So the first thing is be very clear and consistent in your definition of ROI. How is it being calculated? What do you mean by that? Once you've done that, the next challenge is measuring ROI accurately. And the calculation is pretty straightforward, it's incremental sales volume times the profit margin divided by the cost of the marketing. Two of those three pieces are pretty straightforward, but the one piece that's the most challenging to qualify is the incremental volume. And that's where, as we talked earlier, people use various statistic techniques to try to analyze the data and quantify how much incremental volume was truly given by a specific marketing element. You try to distribute the right amount of incremental volume to the right marketing element. And that's where one of the most common techniques is marketing mix modeling. Now that's very prevalent. And that's where one of the other techniques that's prevalent is test versus control. And they each have strengths and weaknesses. And I think people make a common mistake of relying on only one or the other. And you really need the strengths of both. People like marketing mix because, I wouldn't say, it's simple, but it's kind of like one stop shopping. You can throw all of your marketing into the analysis and hopefully get answers for almost all of it. But the challenge is in marketing mix reports on certain performance metrics, like ROI, but again, if you go back to what's the definition of ROI. The definition of ROI in the marketing mix analysis is what I would call a near term ROI or a short term ROI. So it's quantifying the benefit of a marketing execution, the immediate benefit, and maybe a little bit of carryover benefit for 12 weeks maximum. And so if you do a marketing mix analysis and you evaluate various marketing elements, those elements whose objective is to efficiently short term incremental volume, like trade promotion, will tend to score pretty well. And those elements whose objective is a little different and maybe more long-term like equity-based advertising or maybe very sophisticated targeted marketing, where you're only executing against consumers who have not bought your brand before. So you're very focused, you're spending your investment very wisely. You're not wasting anything on anyone who's already bought you because you're trying to acquire new customers. That can be very effective and efficient, that kind of acquisition marketing, targeted acquisition. But the initial benefit of the marketing, it's measured by the marketing mix. It's just a piece of its benefit. It's the initial trial purchase, but the big point of that kind of marketing is you hope to have a good chunk of people who try come back and repeat. And so if you do the ROI customer acquisition marketing over a 12 week window, you get the one number. But if you then look at the ROI of that program over a 52 week window, it could be twice as large. So those are the nuances that need to be considered when you're doing marketing mix modeling and yeah it's a level playing field, but it's only recording one performance metric. That may be the whole picture for some marketing and only part of the picture for other marketing. Then finally, so the first two were we're going to define ROI clearly and then measure ROI accurately. Final one is apply ROI appropriately under making decisions. And then it's back to kind of fundamental blocking and tackling for any given marketing element. Make sure that you're measuring it against its objectives. I already kind of alluded to this with the new customer acquisition marketing. If the objective is to acquire new customers, to get repeat from them, you gotta make sure that you're measuring your increase in penetration. This is another KPI, the amount of trial, repeat rate and things like that, in addition to short-term ROI. So I guess finally, we've talked about the ROI, but there are different types of ROI. There is average ROI, which is what is usually reported as the first number coming out of an analysis. And that's simply, what's the average ROI for the total stand against their marketing element. But there's the marginal ROI, which is what is the benefit of the next dollar that I might spend in that marketing element? So for a marketing element where you spend a lot, you've executed a lot, and maybe you're in the point of diminishing marginal returns. The historic average ROI is not necessarily the number you want to use, when you are looking forward and say, should I spend more on this? So those are some of the things I would keep in mind. >> Wow, that's a lot to take in. Thank you, Paul, for being here with us. Its was good to learn about marketing, return, and investment. And how the growth in data in analytic tools are helping marketers learn more about marketing ROI and improve their marketing effectiveness. I appreciate you taking the time to be here with us and I know we all value learning from your experiences. Thank you for being here with us.
