Hello, we've been talking about threats to internal validity.

And today, we're going to talk about some other factors that are important to

understand, if you're worried about that, and you should be.

So the threats we're going to talk about today are, the testing effects,

something called the John Henry effect, the Hawthorne effect, self-fulfilling

prophecy, halo effect, the placebo effect, and instrumentation effect.

All these you need to be worried about when you're talking about internal

validity, whether you can make a causal influence between your independent

variable and your dependent variable.

So first let's look at testing effects.

We talked about earlier, we talked about learning effects or carry-over effects.

The fact that if you participate in an experiment over several sessions,

then later sessions can be affected by what happens on the earlier sessions.

So they're order effects, basically.

And if you have more than one stage in the experiment,

then you have to worry about those kinds of effects.

These effects are not due to the independent variable, but

they actually carry over between stages in the experiment.

For example, if you have different sessions with the same task,

you sort of learn about the task.

You're practicing the task, and any time you practice something, you do better.

You might have anxiety the first time you end the experiment.

But if you have later sessions, your anxiety is reduced.

And so, is the difference that we see between the sessions

is due to the independent variable?

Or is it just due to the fact that the anxiety of the subject has changed?

We learn to understand what the expectations are for

the experiment with repeated exposures.

So as we go along in the experiment,

we have a better understanding of what the expectations of us as participants.

And that could change what happens,

independent of what's happening with the manipulations in the experiment.

Also, we might have a strategy that we develop so we can do better or

we do worse as we have different sessions.

And finally, we can just have experimental fatigue.

We've been in this experiment now for an hour.

Towards the end of the experiment, performance might be different from what

it was like in the beginning of the experiment,

having nothing to do with the manipulations, just due to fatigue.

So, testing effects are important.

The second kind of threat is that John Henry Effect, and

is based on the old story of John Henry.

He was working in the railroad building with his hammer, knocking down the spikes.

And he heard that they had developed a new technology, a steam

machine that could do this much better and much better than John Henry could do.

When John Henry heard about that, he worked very, very hard,

much harder than he was before he learned this, to compete with the steam machine.

And that could happen in the experiment.

Participants could be, if they identified in the control group,

can work much harder to outperform the people in the experimental condition.

By the way, John Henry died, he worked so hard.

Hopefully, our participants won't.

An example that's actually in the textbook is that

in an educational trial where school classes are in the treatment received

extra support from a support teacher.

The students who are in the control group and don't have the support teacher might

work very hard to overcome the disadvantage that they think they have.

That's just something we have to worry about.

This improved performance of participants because they're comparing themselves

to an experimental treatment.

There's also the Hawthorne effect, an effect that's been around for

a very long time.

In fact, an industrial efficiency study was done

years ago in the Hawthorne Western Electric plant in Chicago.

And they wanted to see whether if they increased the light intensity in

the workplace,

in the lights, that that would lead to better efficiency in part of the workers.

And they found that, in fact, observations that the group felt better and

they worked better when they had the brighter lights.

The problem was they had a low light condition when they lowered the lights and

then a condition where they didn't change anything.

And they worked better and

felt better in the low light condition and the no change light condition.

And what the Hawthorne effect shows is that just the fact that the intervention,

just paying attention to the workers and

observing them, that increase productivity.

Not the independent variable, but just the fact that they were being observed,

that increased their performance.

This is why, in most experiments,

that we do have a treatment that we are using as the independent variable.

We need to have an equal attention to control design, where the same attention

is given to the no treatment condition, as to the treatment condition.

So you can really look at whether the treatment is having an effect,

the Hawthorne effect.

And there's the self-fulfilling prophecy.

This is the idea that, sometimes, by the way, this is called the Rosenthal effect.

We first discussed it, the Pygmalion Effect, but basically it's when

the researcher's expectations actually influence the participant's own behavior.

It's just the fact that if I have an expectation of what I want to

see in the experiment, I can convey that to the participants.

And sometimes it's can occur through body language,

to interactions with the participants.

Having nothing to do with actually intentionally

trying to get the participants to act in a particular way.

It could just happen because of the fact that the researcher wants to know what,

wants to find out what their expectations are.

This can also happen in how the researcher remembers and records the data.

Unintentional, but can change the effect that we

observe in between the independent variable and the dependent variable.

This is why we should need to have "Blind" procedures as control groups

in experiments.

If I'm testing a drug effect, for example,

I want to make sure that the drug that's taken by the participant

does not know whether they're getting the treatment drug or the placebo.

And in fact, I don't know.

That's called a double-blind procedure,

as the experimenter who's getting the drug and who's not.

Then, there can't be these self-fulfilling prophecy provided by

hence from the experimenter to the participants.

The fifth one is the Halo Effect.

And Halo Effect says basically there's a cognitive bias because participants

want to make a impression to the experimenter, person of authority.

And so, sometimes, the person or a company, or a brand or product if you

are doing marketing research, this can influence what the participants will say.

It's a halo effect, it's gotta be good.

There's also the opposite of that, which can have worse

feelings about something because that's what they expect it to do.

So, it's an irrelevant feature, like the attractiveness of the person,

how the person is dressed, the experimenter.

All that could influence what happens in the experiment because there's

positive and sometimes negative direction that the participants will see

in the experiment themselves.

But 6 is a very common one used a lot in drug research and

that's the placebo effect.

And a placebo is a control intervention which has all the characteristics of

the intervention except for the active ingredient.

The thing you really are studying is the independent variable.

These are used a lot, as I mentioned, in drug experiments.

The placebo is a substance that looks just like the drug, smells like the drug,

tastes like the drug.

But it doesn't have whatever that active ingredient is that's supposed to make

the drug work.

And placebo effect in behavioral research is very powerful.

In fact, placebo effect is sometimes are than 30% percent

of the actual intervention effect.

So let's take a hypothetical study again, it's one in your text readings,

where the treatment condition and placebo condition are given.

And the results are that the participants expect to improve with the placebo or

with the treatment actually improved over the no treatment condition,

which doesn't have, nothing happens.

They get neither the placebo or the active drug, but there is this placebo effect,

the red versus the purple bar that's very dramatic.

And we can misinterpret what's a difference between the no treatment and

the treatment condition as having an effect,

when in fact it might not, when we include a placebo condition.

So if there was no placebo control, we might assume that the treatment worked,

if there is a placebo, we can really tell whether the treatment is working.

7, the instrumentation effects, and this is where if I have to change something in

an experiment like change the survey because I asked so many questions.

I changed the location where I'm testing, I changed the computer program that

actually presents the materials to the participants.

These different measuring devices might have different precisions.

And so we get different effects.

It might have a different ceiling effect or different floor effect.

That is, I might not see a difference because

the measuring device doesn't allow me to see larger differences.

And not see a difference because the new measuring device

doesn't allow me to see lower performance levels.

The instrumentation effect,

we gotta keep things constant if we change something, that change,

even though it's unintentional, might have an effect on the experiment.

So threats to internal validities.

Other threats that we talked about, we talked earlier about maturation.

There might be change in the people themselves as they go through

the experiment.

History, things could happen in the environment

that could affect their performance.

If you were tested the day after 9/11,

when the airplanes flew into the World Trade Center in New York City USA,

that might affect your performance on the experiment.

It's very different from just if you hadn't had that experience.

Regression to the mean, the fact if you have an extreme score, and

then you test again.

By just statistical happening, it's going to get closer to the mean.

We've looked at attrition and mortality.

If we have a study that lasts a long time, people will drop out because of that

experimental fatigue or because of just lack of interest and some people die.

If it's a long study, like an aging study,

the old group might have fewer people and those fewer people might really make

a difference between the participants in the two groups in the experiment.

We talked about carry-over effects and testing effects, the John Henry effect,

the Hawthorne effects, self-fulfilling prophecy,

halo effects, placebo effects and instrumentation changes.

All these are things that could influence the relationship between an independent

variable and dependent variable, but ones that we want to try to avoid at all costs.

Thank you.

Lesson 5D: Testing Threats

Experimental Research Methods in Psychology

Meet the Instructors