One of the biggest issues you're going to to deal with if you're going to start
doing predictive analytics is dealing with some of the organizational issues.
These can be politics.
These can be technology.
But ultimately remember what we're doing.
We're taking data, big data if you have it, we're estimating some models.
That requires technology.
And then based on those models, I'm going to start changing my organization.
Guess what, some people are going to win, and some people are going to lose.
You may find out that an intuition or a hypothesis you had is completely wrong.
The person who has basically made their career based on that
intuition is not going to be real happy on this.
So here are some of the issues that you're going to need to think about if you're
going to start doing predictive analytics.
Okay, some of the common reasons why companies have trouble doing this, right.
Insufficient model development, right.
You can't do data mining, which has a lot of benefits, doing data mining,
but ultimately what you want to know is, right, what leads to what?
And what actions can I actually take as a company to do this?
So even if you do the data mining, you need to, again, start doing this peeling
back the onion, which means you need to think about what these linkages are.
From my strategy, what do I think this causal business model is?
How is it that I'm going to implement this?
Well, I think our A is going to lead to B is going to lead to C.
Let's see, but you need to lay that out, right?
The other problem I've seen in a lot of companies is, they will ask me, okay,
what's best practices?
Or, I saw this benchmarking model or this generic measurement framework,
like the balance score card.
So I'm going to use that to pick my performance measures.
Well, there's problems with that.
Again, strategic advantage means you're doing something different than your
competitors.
So the last thing I want to do is benchmark myself and
do exactly what they're doing.
The other problem with generic measurement frameworks, they're generic,
which means it says, okay, everybody should be doing this.
Now what you want to do is tailor both the measures that you track and
the analyses you do to the strategy of your company.
So if I think I'm going to compete on a different dimension I do not want to do
what the other guy did.
So this means there's going to be a lot of up front, starting with your strategy,
to decide, here's the analytic models I want to estimate.
It's not purely statistics.
And based on that, what are the actions that we as a company can take?
Another problem you have is we are taking measures, right.
Measures can be good.
Measures can be really bad.
A couple of reasons why you might have measures that are bad are what are called
psychometric properties.
One, does it really pick up what you claim it's picking up, right?
You're trying to estimate some construct and
I gotta find some way of measuring this kind of intangible thing.
Well, is that measure you have,
is that really picking up that intangible you care about?
And the other thing is, is this influenced by so
many other things that it bounces up and down all of the place?
And I have no idea whether that means you're doing well or not doing well.
Now, this really becomes a problem when you start using surveys for questions.
One of the problems is you may have too few questions, right.
Like, how satisfied are you?
A little, a lot? And that's the only question I ask you?
That doesn't help me much afterwards because even if I find a relationship,
I don't know which dimensions of satisfaction you answered about, and
I don't know what to do as a manager.
Another one with scales is you have too few scale points, right,
you're using one to three.
I'm not satisfied, I'm very satisfied, or I'm in the middle.
That doesn't really tell you a lot.
Or you use what's called the top-box measure.
Say you're satisfaction scale goes from one to five.
What percentage of the people are at five?
That may be fine, as we saw in some of the examples.
In other examples it might not be fine.
You need to do the analysis of whether moving everybody to the top of the box
even makes any sense.
But again, what you would like to have
are measures that have what's called good signal to noise ratio.