Linear models, as their name implies, relates an outcome to a set of predictors of interest using linear assumptions. Regression models, a subset of linear models, are the most important statistical analysis tool in a data scientist’s toolkit. This course covers regression analysis, least squares and inference using regression models. Special cases of the regression model, ANOVA and ANCOVA will be covered as well. Analysis of residuals and variability will be investigated. The course will cover modern thinking on model selection and novel uses of regression models including scatterplot smoothing.
Multivariable Regression part I
Loading...
From the course by Johns Hopkins University
Regression Models
2019 оценки
From the lesson
Week 3: Multivariable Regression, Residuals, & Diagnostics
This week, we'll build on last week's introduction to multivariable regression with some examples and then cover residuals, diagnostics, variance inflation, and model comparison.
Meet the Instructors
Brian Caffo, PhDProfessor, Biostatistics
Bloomberg School of Public Health
Roger D. Peng, PhDAssociate Professor, Biostatistics
Bloomberg School of Public Health
Jeff Leek, PhDAssociate Professor, Biostatistics
Bloomberg School of Public Health
Hi, and welcome to the class on multivariable regression,
as part of our course here at Data Science Specialization.
In this lecture,
we're gonna talk about instances where we have lots of potential predictors for
an outcome, rather than just a single predictor in linear regression.
In any instance, where you're using a predictor X to predict a response Y,
and you find a nice relationship, if that predictor
hasn't been randomized to the subjects or units, whatever you're looking at,
then there's always a concern that there's another variable that you either know
about or don't know about that might explain the relationship.
And let me give you an example.
Imagine if you had a friend that downloaded some data, where they had all
sorts of health information from people and also their dietary information.
And this person said, hey, I found something interesting.
Breath mint usage has a significant regression relationship with forced
expiratory volume, a measure of lung function, pulmonary function.
If you've ever gotten an asthma test, they'll measure your FEV.
Well, you would be skeptical.
I mean, there's very little basis for a biological relationship there.
Breath mints are just sugar.
It doesn't seem reasonable that they would impact lung function.
But maybe, but what you've really be thinking is,
well, what other variables might be explaining this relationship?
And you might come up with two hypotheses.
One is, this person dug through lots and lots and lots of variables and
just found the one that was significant, and it's just a chance of association.
And that's the problem of multiplicity.
Okay, so we'll talk about that in other aspects of this course in the inference
course, but let's just assume that this person didn't do that, they looked only
at a couple of variables, and the multiplicity concerns weren't so bad.
Then what would you think?
Well, likely, you would think,
well, probably the real problem is smokers tend to use more breath mints, and
smoking has this long relationship with lung function, so
it's well-established that chronic exposure to a smoker,
even second-hand smoke has negative impacts on lung function.
So it's probably that, it probably has nothing to do with the breath mints,
it's a indirect effect of breath mints through smoking,
not a direct effect of breath mints on lung function.
That would be your likely hypothesis.
Well, how would you establish that there's a breath mint effect beyond smoking?
Well, likely, what you would ask your friend to do is,
well ,consider smokers by themselves, and see whether their lung function
differs by their breath mint usage, and consider non-smokers by themselves, and
see whether their lung function differs by breath mint usage,
where you've conditioned on smoking status so that you're comparing like with like.
Well, multi-rate regression is sort of automated way to
do that in a linear fashion.
It make assumptions, which is fair enough, but it does that sort in automated way,
and we'll explain in this lecture in what way is it trying to sort of
hold smoking status constant while looking at breath mint usage,
and how it adjusts, and we'll also talk a little bit about its limitations.
But that's the fundamental idea of what multivariable regression is trying to do.
It's trying to look at the relationship of a predictor and
a response, while having, at some level, accounted for other variables.
So in addition to the use of multivariable regression that I talked about in
the previous slide,
I wanted to also talk about another use of multivariable regression for prediction.
It's actually a very good prediction model.
So, as an example, several of us engaged around here engaged in this so
called CAGO competition, to predict the number of days that a person
would be in the hospital in subsequent years given their claims history and
number of days they were in the hospital in previous years.
And this is of major importance to hospitals and
insurance companies and healthcare providers for a variety of reasons.
It's one of the main questions in that field.
So, in this competition, they gave you historic claims data,
a lot of it, actually, which had lots and lots of predictors, and
the number of days that several people of the insured people from
this company were in the hospital for [NOISE] three consecutive years or so.
And simple linear regression certainly wouldn't have been too much use here.
It's good for looking at variables at a time really quickly, but
if you wanted to get good prediction you needed to do something more complicated.
So there's multiple parts about this, so one of the main things is model search.
How do we pick which predictors to include?
That's an important component of this.
And the other is to avoid overfittings.
We'll learn that, as you put enough variables into a multivariable
progression model, you'll get zero residuals just by virtue of having
included even random vectors into your regression model.
So certainly there's consequences to throwing lots of garbagey
predictors into a model.
And certainly there must be consequences to omitting
important predictors in a model.
And in the practical machine learning class,
which is also part of the specialization, you will learn a lot about model
selection strategies as they relate to the idea of prediction.
In this class, we're gonna focus more on the problems from the previous slide,
where we want to generate parsimonious models, where we're
deeply interested in interpreting the coefficients from the linear model.
And so the prediction problems, like this one,
are a little bit more geared toward our practical machine learning class.
But I just wanted to mention that multivariable regression is
a pretty good starting point in any prediction,
anytime where you're developing a prediction algorithm.
What we found in that competition is that multivariable regression got you very
close to the winning entry, and
lots of machine learning, and random forest, and boosting, and
all these other things, those only got you minor bumps on top of that.
And so, to get huge drops in prediction error,
well-thought out linear models sufficed, and
then to get really minor increments beyond that, you had to throw a lot of computing.
And to be fair, those did improve your chances, and we moved up
a little bit in the leaderboard by adding some of these more complicated things.
But it was remarkable how far you could get with just well done linear models.
So, our linear model looks an awful lot like our simple linear model.
It's just that we have more variables.
So our outcome, Y, it might be insurance claims or
forced expiratory volume, is equal to a bunch of coefficients.
These are the beta terms, they're like the slope terms in simple linear regression.
Just, now there's more predictor terms, more X values.
So, for example, one of these might, X1 might be breath mint usage of binary or
variable, and X2 might be number of pack years or how much a person smoked.
And in the insurance case, X1 might be the number of
insurance claims in the previous year, and X2 might be whether or
not this person had a particular cardiac problem, something like that,
that might lead toward information about hospitalization in the successive year.
So here, we just write out the linear model.
And it just looks like the outcome is equal to a bunch of coefficients
times predictors.
Now, it's linear because it's linear in the coefficient.
So I'll reiterate that point in a minute.
I would also add that the first variable is typically just a constant one,
so there's an intercept that's included, a term that's just beta by itself.
Beta zero usually or Beta one.
So our least squares,
I think everyone could probably guess what the least squares is going to do.
It's just gonna look at the differences between the outcome and the prediction
from the linear models, summation of the predictors times their coefficients.
Because that's not necessarily positive,
we're gonna square it and we're going to add it up so this difference for
every observation equally weights into this loss function that we created.
So least squares simply wants to minimize this equation.
And it's a direct extension of the equation we wanted to minimize
when we had simple linear regression.
And I would notice that the important linearity is linearity in the coefficient.
So for example, if I take one of my X's and just square it, meaning
if I have a vector in R that I've just squared every element of that vector and
put that in as part of my model, then that will still be linear in the coefficient.
The coefficient won't be squared, it will just be the X term.
And so, the important version of linearity is linearity in the coefficient.
That's what defines a linear model.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
