5.7 Type II errors

Loading...

From the course by The Hong Kong University of Science and Technology

Social Science Approaches to the Study of Chinese Society Part 2

3 оценки

The Hong Kong University of Science and Technology

Social Science Approaches to the Study of Chinese Society Part 2

3 оценки

This course is intended as a first step for learners who seek to become producers of social science research. It is organized as an introduction to the design and execution of a research study. It introduces the key elements of a proposal for a research study, and explains the role of each. It reviews the major types of qualitative and quantitative data used in social science research, and then introduces some of the most important sources of existing data available freely or by application, worldwide and for China. The course offers an overview of basic principles in the design of surveys, including a brief introduction to sampling. Basic techniques for quantitative analysis are also introduced, along with a review of common challenges that arise in the interpretation of results. Professional and ethical issues that often arise in the conduct of research are also discussed. The course concludes with an introduction to the options for further study available to the interested student, and an overview of the key steps involved in selecting postgraduate programs and applying for admission. Learners who complete the course will be able to make an informed decision about whether to pursue advanced studies, and should be adequately prepared to write an application for postgraduate study that exhibits basic understanding of key aspects of social science research paradigms and methodologies. Explore the big questions in social science and learn how you can be a producer of social science research. Course Overview video: https://youtu.be/QuMOAlwhpvU Part 1 should be completed before taking this course: https://www.coursera.org/learn/social-science-study-chinese-society

From the lesson

Quantitative Analysis

Week 5 will give you a taste of the basic methods for quantitative analysis. From there you should be able to identify key issues when interpreting results and discuss implications for research.

5.1 Overview: Quantitative Analysis3:02

5.2 Tabulations10:53

5.3a Correlation and regression I12:06

5.3b Correlation and regression II6:18

5.4 Regression to the mean18:53

5.5 Statistical significance12:45

5.6 Type I errors9:31

5.7 Type II errors7:29

5.8 Summary Quantitative Analysis3:14

Meet the Instructors

Cameron Campbell
Professor of Social Science
Associate Dean of the School of Humanities and Social Science

Hi, in this module I'm going introduce type II errors to you,

or for some of you this may be a review.

Type II error refers to the phenomenon of incorrectly concluding that there

is no relationship or no effect when in fact there is one.

In other words, we fail to reject the null hypothesis even when we should reject it.

So it's the opposite of the Type I errors that we just talked about.

A type I error, again,

is a situation where we have a false positive.

We conclude that there is a relationship,

a effect, even more,

even when there actually isn't one.

So here, there is an effect,

or there is a relationship,

but we don't detect it.

So we failed to deduct a difference or

relationship that actually exists out there in the population.

If only we could measure it properly.

Statistical power refers to the ability to detect a difference or a relationship.

You'll hear the term 'statistical power' quite often,

especially if you take a statistics class taught in

Psychology or Epidemiology or certain other fields where they

are more concerned or more adept at dealing with type II errors.

And you'll often see sometimes people write a Beta,

the Greek letter Beta,

as shorthand for the probability of a type II error.

So, one minus beta is our statistical power.

So just to give you a few examples of what a type II error might be.

Maybe a situation where we're testing the effect of some drug,

and actually maybe the drug does have an effect.

But when we do our test,

we don't find that effect,

or there is a relationship between, say,

education and income that we don't pick up or at

least we don't regard as statistically significant.

So I mentioned that one minus Beta is the measure of statistical power.

So quite often, again,

if you take statistics from psychologists or in certain other fields,

we'll talk a lot about this because they get to design

their studies themselves and they

decide on how many subjects they're going to include in their studies.

So they figure out in advance what kind of statistical power they

need or what level of significance they want to be able to discern and affect that.

And then they carry out what are called 'power

calculations' to work out the size of the study,

the number of subjects they'll need in order to properly

resolve whether or not an effect exists and therefore,

hopefully, avoid a type II error.

Now, how do people,

when they're worrying about this,

reduce the chances of a type II error?

Well, they can increase the sample size,

that increases statistical power.

So we mentioned that a larger sample size

generally yields larger test statistics which reduce our P value.

So, again, less likely do commit a type II error with a larger sample,

less likely do miss an effect that actually is there.

If we are working with secondary data where we don't have any control over

the size of the sample because somebody else collected the data,

and we inherited it or accessed it,

we can try pooling data.

So increasingly some people try to pool data from, perhaps,

multiple waves of the same study or from multiple studies that cover the same topic,

and that gives them more cases, more statistical power.

Power that helps them detect possibly relationships that they might otherwise miss.

They can always loosen the criterion for the hypothesis tests,

so you settle for a criterion of point five for the significance instead of point 01.

Now, that may be a bit risky because of course it increases your chances of a type I error

when you loosen the criterion for the test.

And again, something that's always

a good idea you can try to reduce the measurement error.

So a measurement error,

variability in the measurement of the outcome,

and the other variables, can,

by introducing noise into our measurements,

change or alter our test statistics and

make it more likely that we'll end up committing a type II error that is,

or missing effect that actually is there.

So, what are some of the implications of

the prospect or the possibility of type II errors?

One important one, and I see people

violating this principle actually on a fairly regular basis,

is that statistical significance, it helps prove,

that or at least helps provide evidence that a relationship or a difference exists.

But the reverse is not true.

Lack of statistical significance does not rule out the existence of a relationship.

So, lack of statistical significance should be interpreted as

a relationship unproven or relationship not proven in Scottish jurisprudence,

but it's not a relationship disproven.

So sometimes I see people making presentations and they'll report,

say regression coefficients, that actually have large magnitude,

or effect sizes that appear very large but actually

the P values make it clear that the effect is not statistically significant.

And then they go on to say, 'well, in fact,

the result is not statistically significant,

therefore, there is no relationship'.

So again, a large P value is not a proof that a relationship doesn't exist,

it just means that the data at hand is not sufficient to prove that it exists.

There may be a relationship but our sample may be too

small or there may be too much noise or our criterion,

in some cases, may not be too strict.

I wouldn't advise loosening criterion but never hurts to try to gather more data.

So, that would be the one thing that I would ask you to keep in mind.

On the one hand, if you are going into psychology or

certain other fields where you design studies yourselves, experimental studies,

you'll have to learn about statistical power,

about power calculations to work out what kind of sample size you're going to

need to measure the effects that you hypothesize are there.

But for the rest of us,

I think the practical implication of thinking about

Type II errors in the meaning of P values,

especially values that don't appear statistically significant,

is that if something isn't statistically significant

that doesn't prove the relationship isn't there,

i it just means we can't reject

the null hypothesis and we don't have evidence of a relationship.

You want to disprove a relationship,

if you want to rule out a relationship,

need to show that there is a small magnitude or a small coefficient size,

and you have to have a large sample to convince

people that you're really decisively ruling it out.

Ознакомьтесь с нашим каталогом

Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.

Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.

© Coursera Inc., 2018 Все права защищены.

Загрузить из App Store

Загрузить в Google Play