Explore
For Enterprise
Join for Free
Log In

Гуманитарные науки и искусства
Бизнес
Компьютерные науки
Наука о данных
Информационные технологии
здоровье
Математика и логика
Личное развитие
Естественные и технические науки
Социальные науки
Изучение языков
Степени
Сертификаты

Полный обзор Coursera

Обзор
Поиск
Для организаций
Войти
Присоединиться бесплатно

Estimating the Difference Between Two Proportions

Loading...

Статистика вывода

Университет Дьюка

4.8 (оценок: 1,435) | Зарегистрировано учащихся: 54K

Курс 2 из 5 — Statistics with R Специализация

This course covers commonly used statistical inference methods for numerical and categorical data. You will learn how to set up and perform hypothesis tests, interpret p-values, and report the results of your analysis in a way that is interpretable for clients or the public. Using numerous data examples, you will learn to report estimates of quantities in a way that expresses the uncertainty of the quantity of interest. You will be guided through installing and using R and RStudio (free statistical software), and will use this software for lab exercises and a final project. The course introduces practical tools for performing data analysis and explores the fundamental concepts necessary to interpret and report results for both categorical and numerical data

Просмотреть программу курса

Получаемые навыки

Statistical Inference, Statistical Hypothesis Testing, R Programming

Рецензии

4.8 (оценок: 1,435)

5 stars
1,195 ratings
4 stars
201 ratings
3 stars
28 ratings
2 stars
7 ratings
1 star
4 ratings

DS

Mar 08, 2017

This course is an excellent overview of inferential statistic tests / hypothesis tests and confidence intervals. The organization and material is quite good, with exercises and applications using R.

RR

Jun 15, 2017

Awesome. I loved the way this course is done. I know what Test Statistic to use for what type of data and under which conditions. I am preparing a cheat-sheet that will be shared with all later on.

Из урока

Inference for Proportions

Welcome to Week Four of our course! In this unit, we’ll discuss inference for categorical data. We use methods introduced this week to answer questions like “What proportion of the American public approves of the job of the Supreme Court is doing?”.

Introduction3:40

Sampling Variability and CLT for Proportions15:43

Confidence Interval for a Proportion9:59

Hypothesis Test for a Proportion9:27

Estimating the Difference Between Two Proportions17:24

Hypothesis Test for Comparing Two Proportions13:39

Преподаватели

Mine Çetinkaya-Rundel
Associate Professor of the Practice

Текст видео

Now we extend our discussion on categorical

variables,from working with only one categorical variable

at a time, to working with two

categorical variables, and evaluating the relationship between them.

So in this case, we're going to label one of

our categorical variables the explanatory variable, or the grouping variable.

And the other one, our response variable.

And so, were working with categorical variables that have two levels,

so a success and a failure is what were going to

be categorizing them as.

Were going to calculate proportion of successes in the two

groups ,based on our sample, and were going to compare this

to each other to be able to say something about how

the proportion of successes, in the population compared to each other.

And since we're calling this an estimation, as

usual we're going to use a confidence interval.

In other words, what we're going to do in this video

could be summed as, calculating a confidence interval for the difference

between the two population proportions ,that

are unknown using data from our sample.

Here is the data set that we will be working with.

In early October 2013, a Gallup poll asked,

"Do you think there should or should not be

a law that would ban the possession of

handguns, except by the police and other authorized persons?"

The possible responses on the survey were no, there should not be such a law,

yes, there should be such a law, or no opinion.

And we're going to categorize ,yes there should be such a law as success, and we're

going to categorize either no opinion ,or no

there should not be such a law as failure.

Let's take a look at the survey results from Gallop first.

Gallop actually has been asking this question.

On support for ban on possession of hand guns, since 1959.

And we can

[BLANK_AUDIO]

Look to see how things have changed in the U.S. since then.

The dark green line represents the percentage of respondents

saying, no there should not be such a law.

And the light green ,is the percentage of respondents

that say, yes there should be such a law.

So back in the 1950s or early 1960s, the case was that there

were higher proportion of people thinking yes ,there should be such a law

and, today that's a much lower proportion at 25%.

Compared to the 74 that say no there should not be such a law.

74 plus 25 makes 99% so that 1% missing must have been

the proportion of people who said they had no opinion on the matter.

We also asked this question to a group of

Coursera students as well, and here is what the responses

looked like.

Of the 83 responses we got [INAUDIBLE], 59 people,

so 71%, said yes, there should be such a law,

and 24%, 20 out of 83, said no there should not be such

a law, and 4 out of 83 said that they had no opinion.

This seems like a very different population and that's probably expected.

Coz there is a much more international population

than the US and ,possession of guns and what

not tends to be a hot topic in the US, may not necessarily be so in other countries.

So this is what we're seeing

as a distribution.

We can actually take a look at our results summary.

So in the U.S. Gallup had surveyed 1,028 people and

25% of them said yes there should be such a law.

That makes 257 people.

And within the Coursera population we had a sample of 83 respondents.

And the proportion of success, so [INAUDIBLE] respondents

that said yes there should be such a law is 71%, so that's 59 out of 83.

Using these values, we want to answer the

following question, how do Coursera students and the

American public at large compare with respect to

their views on laws banning possession of handguns?

So our parameter of interest is the difference

between the proportions of all Coursera students ,and

all Americans who believe there should be a ban on possession of handguns.

So that could be, P of Coursera minus P of

U.S. P stands for the

population proportion, or unknown population parameter.

And our sample statistic, that serves as a aa point estimate for this

parameter of interest, is the difference between

the proportions of sampled Coursera students and

sampled Americans ,who believe there should be a ban on possession of handguns.

And that if, that could be denoted as P hat of Coursera, minus P hat of US.

So how do we estimate differences between proportions?

Well, estimation we said is usually a confidence interval.

And a confidence interval is always of the form

point estimate plus or minus a margin of error.

In this case our point estimate ,is

the difference between two sample proportions ,and our margin

of error as usual, can be calculated as a

critical score, this is a critical Z score, we're

dealing with sample proportions ,and based on the central limit

theorem, we know that the distribution of those are

going to be nearly normal ,for the difference between the

sample proportions as well, so we're using a Z

score, for our critical score, times we have the standard

error of the difference between the two sample proportions.

So once again, the new concept here is, how do we calculate the

standard error ,for the difference between two

proportions, for calculating a confidence interval .

We're going to get to in a little bit why

we're, you know, specifying for calculating a confidence interval.

But ,we've already seen this before with the one proportion case, as well.

So, there's going to be a difference

between how we calculate this

for confidence interval versus hypothesis tests.

This video focuses on confidence intervals.

The next one will walk you through how to do this as [INAUDIBLE] a hypothesis test.

So the standard error ,for the difference between the two sample

proportions ,can be calculated as the square root of P hat 1.

That's the proportion of successes in group one

,times 1 minus P hat 1, that's the

proportion of failures in group 1.

Divided by that group sample ,size and then

the same quantity for the second group as well.

Important thing to mention here are that

these are P hats ,so observed sample proportions.

We add up these two quantities ,just like

working with means, we're bringing two unknowns together, a

P hat one and a P hat two ,sample

proportions from two groups ,so the overall variability should

increase and hence we're adding the two Variability components here,

and then finally taking the square root of that, to go

from variance, to standard deviation, or in other words, the standard

error, which is basically the standard deviation of the sampling distribution.

Of course, we just introduced a new formula, that

means we need to go through the conditions as well.

The first condition, as usual, is independence.

Within the groups, we want to make sure that sampled observations

are independent of each other.

And we can ensure this using a random sample or

assignment, depending on whether we have an observational study or an

experiment ,and if we are sampling without replacement, we want to

have our sample size be less than 10% of our population.

And between groups, we want to make sure

that the two groups are independent of each other.

In other words, the data are not paired. From means, we had

actually introduced a method for working with

paired means, but for proportions, we're only

going to focus on ,methods that work

with Independent groups for comparing two proportions.

We also need some sample size conditions ,to ensure normality of the

sampling distribution ,and this is that

each sample should meet the success-failure condition.

So we want to meet the success-failure condition for the first sample as

well as the second sample.

And remember that if we're doing

a confidence interval, these proportions that we're

considering to calculate the success-failure condition

need to be our observed sample proportions.

So, let's put all of that to use.

Using a 95% confidence interval, let's estimate ,how Coursera

students and the American public at large ,compare with respect to their views on

laws ,banning possession of handguns.

And we have our data here as well, but before we can move

on to do doing the confidence interval, we need to check our conditions.

Independence condition.

We do have a random sample ,for the

US population, but not for the Coursera population.

This was simply a voluntary poll that was posted on the Coursera discussion forums,

so we can't say that this is a random sample.

So, while Gallup does a good job of getting a random sample from

the US, we did not, certainly did not do so for the Coursera students.

And for the 10% condition though, that's math for both of the groups, because both

1028 and 83 are less than 10% of their respective populations.

What does that mean? The sampled Americans

,can be assumed to be independent of each other.

However the sampled Courserians may not be.

So in this case, we want to be kind of

wary of [INAUDIBLE] generalizing any of our conclusion from these

findings ,to the overall population at large ,because we

don't really have a good sample from the Coursera population.

We're still going to move on with

our analysis only for illustrative purposes those.

So, in this case because we don't have a good sample, we want to make sure

that we're being very very careful about generalization

at the end of the confidence interval calculation.

The other condition we need to meet ,is the sample size and skew condition.

So this is about, whether we can assume the sampling distribution

of the difference between the two proportions to ,be nearly normal.

And when we're dealing with proportions ,we

check for this using the success failure rule.

And when we're doing a confidence interval we check

for this using the observe successes and observe failures.

In the US ,we had 257 observed successes and

1,028 minus 257, 771 observed failures.

Here what we mean by observed successes ,is those who voted for yes,

there should be a ban on possession of handguns, and the remainder

are those who said no, there should not be or no opinion.

For the Coursera students ,we have 59 successes ,and 24 failures, so it

appears that both of these are met, the suc success failure condition is met.

For both of the groups and therefore we can assume.

That the sampling distribution ,of the difference

between ,two proportions is nearly normal.

Now that we got our conditions out of the way.

We can actually calculate our confidence interval.

That's a point estimate so the difference between two sample proportions.

Plus or minus some margin of error, that we

calculate as a Z star, times the standard error.

The sample proportion for Coursera students is .71, and

the sample proportion for Americans is .25, so, our point

estimate is the difference between them, since we are

asked for a 95% confidence interval, the Z star

is 1.96, and we can calculate our standard ,error

as the square root of the sum of two quantities.

So, for the first quantity, we're looking at

the proportion of successes in Coursera times the

proportion of failures divided by the sample size

for the Coursera students ,plus the proportion of

the successes in the US times the proportion of

failures divided by the sample size for the US.

[INAUDIBLE].

The standard error ,then comes out to be .0516 and based

on this, the margin of error comes out to be roughly 10%.

In this case, our overall confidence interval

comes out to be between .36 and .56.

What this means is that we are 95% confident ,that

the proportion of Coursera students who believe that their should be a ban on

possession of handguns ,is 36 to 56%

higher than the proportion of Americans who do.

That's a huge difference even when we

factor in the variability ,around the point estimate.

And, again, this is probably expected based on

how different the composition of the two populations are.

So, we were asked to estimate the difference ,and we made an executive

decision to put the Coursera student

proportion First, and the American proportion second.

But does the order actually matter?

So does our decision actually change things?

The answer is yes and no.

It might change some of the calculations along the

way, but it's not going to change your conclusions.

Remember, this is how we calculate our

confidence interval for the difference between two proportions.

The point estimate ,plus or minus a margin of error.

Your margin of error is bound to be a number that's always, always positive,

because if you think about it, the

standard error is always going to be positive.

Conceptually speaking negative variability doesn't

make sense, and mathematically speaking, the

standard error is a quantity that is calculated as a square root.

And your Z star by definition is

always going to be positive as well.

On the other hand, your point estimate can be positive or negative.

Depending on in which order you're subtracting things.

In our calculations, we chose to put Coursera first ,and US second.

So we were looking for the difference between

the proportions of Courserians, all of them, and Americans.

All of them who believe that possession of hand guns should be banned.

And because we don't know these true proportions,

we decided to estimate this using a confidence interval.

In construction of our confidence interval, since

we put Coursera first, we first plugged

in the sample proportion for that group,

subtracted from that the sample proportion for Americans.

Added and subtracted the margin of error.

This gave us .46 plus or minus .10 which yielded a confidence interval of .36

to .56. How do we reverse things though?

The second line of the calculation would be only slightly different.

It doesn't matter in which order you calculate the standard error.

You would still get the exact same standard

error ,and hence the exact same margin of error.

So, the .10 stays, but the point estimate ,would

now be calculated as .25 minus .71, which would

yield negative .46 plus or minus .10, which is going

to yield what looks like a slightly different confidence interval.

So, this goes from a negative .56 to negative .36.

Well, I think we can see a pattern here.

What we're seeing here is that ,the bounds of the confidence

interval are now flipped, and their signs are flipped as well.

So,

instead of .36 to begin with, we have negative .36 to end with, and

instead of .56 to begin with, we have negative .56 to end with.

So, what's going on here?

Well, these are actually exactly the same thing.

The first interval could be interpreted as we are 95% confident that the proportion

of Coursera students ,who believe there should be a ban on handguns is ,36%

higher to 56% higher than the proportion of Americans who do.

Or we could rephrase the same thing as the proportion of Americans.

Who believe there should be a ban on han, possession of handguns is, 56% to

36% lower, than the proportions of Courserean's.

So, it's just which one you put first and which one you put second in your

interpretation, but as long as your interpreting correctly ,it

doesn't matter in which order you do the calculations.

We're still going to get the same thing, except we're

going to make up for it in our interpretations.

Now, one last step working with this confidence interval.

Based on the confidence interval we calculated, should we expect

to find a significant difference at the equivalent significance level.

Between the population proportions of Coursera

students and the American public at

large ,who believe there should be a law banning the possession of handguns.

And remember our confidence interval was from

.36 to .56.

What we're asked to do here is kind of a

mock hypothesis test, and in that case, our null hypothesis would

be That the difference between the two population proportions is

0, because that basically says that no, there's nothing going on.

These two populations are exactly the same.

This is what our confidence interval would look like on a number line.

Anything between .36 and .56 is fair game ,for

a difference between the two population proportions.

In that case ,the value zero ,for the difference between

the two population proportions does not appear to be fair

game and ,based on that we would actually reject this

null hypothesis ,and say that based on this confidence interval.

It doesn't appear that these two populations are the

same with respect to proportion of those who believe there

should be a law banning the possession of handguns.

Ознакомьтесь с нашим каталогом

Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.

Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.

© Coursera Inc., 2019 Все права защищены.

Загрузить из App Store

Загрузить в Google Play

Coursera
О проекте
Руководство
Карьера
Каталог
Сертификаты
Сертификаты MasterTrack™
Степени
Для организаций
Для правительственных организаций

Сообщество
Learners
Partners
Developers
Beta Testers
Translators

На связи
Блог
Facebook
LinkedIn
Twitter
Instagram
YouTube
Технический блог

Еще
Условия
Конфиденциальность
Помощь
Доступность
Пресса
Контакты
Справочник
Филиалы