This course presents research conducted to increase our understanding of how data collection decisions affect survey errors. This is not a “how–to-do-it” course on data collection, but instead reviews the literature on survey design decisions and data quality in order to sensitize learners to how alternative survey designs might impact the data obtained from those surveys. The course reviews a range of survey data collection methods that are both interview-based (face-to-face and telephone) and self-administered (paper questionnaires that are mailed and those that are implemented online, i.e. as web surveys). Mixed mode designs are also covered as well as several hybrid modes for collecting sensitive information e.g., self-administering the sensitive questions in what is otherwise a face-to-face interview. The course also covers newer methods such as mobile web and SMS (text message) interviews, and examines alternative data sources such as social media. It concentrates on the impact these techniques have on the quality of survey data, including error from measurement, nonresponse, and coverage, and assesses the tradeoffs between these error sources when researchers choose a mode or survey design.
3.1.2 Respondent selection, Within Household Sampling
Loading...
Из курса от партнера University of Michigan
Data Collection: Online, Telephone and Face-to-face
127 оценки
Из урока
Module 3: Interviewers and Interviewing
In this lesson, we explore the various roles interviewers take on beside asking questions and collecting answers, as well as some of the different approaches to interviewing that have been proposed and how they affect the accuracy of responses. By the end of Lesson 3, you will appreciate the benefits and costs of collecting data in interviews and will be able to contrast them with the costs and benefits of self-administration.
Познакомьтесь с преподавателями
Frederick Conrad, Ph.D.Research Professor, Survey Methodology
Institute for Social Research
Respondent selection, as I mentioned before,
this is a kind of extra step in the sampling process that comes
about often when the units in the sampling frame don't match
the units of analysis that the study is oriented towards.
So that if for example, the frame lists households say,
street addresses or landline telephone numbers, but
the survey is about Individuals, as is often the case.
Then this extra layer of sampling is necessary within the household to pick
just one member to be the respondent from among all the household members.
There are a number of different techniques for doing that.
We won't go into all of them.
Just to touch on a few of the well known ones, and a hybrid one,
the best known is the Kish method, which is generally considered the gold standard.
It's actually the oldest.
It produces by far the most random sample and
assures equal probability of selection.
It's also considered to be the most intrusive because it requires
respondents or
household members to reveal considerable information about the household.
The birthday methods are much simpler.
They simply involve asking who in the household had the last birthday or
the next birthday.
They're much simpler, but
they don't necessarily reveal the probability of selection.
Because they don't require the number of people in the household to be
indicated by the respondent.
So there's a trade-off, but they are simpler to administer.
And then the Rizzo, Brick, and Park model actually uses either
the birthday method or Kish and makes use of just sort of coin toss techniques.
So there are a number of other approaches to selecting a person within
a household besides using one of these selection methods.
There's, of course, no sampling,
just interviewing the person who opens the door or answers the phone.
And you can give some thought to why this might bias the sample.
You're invited to post your thinking about this in one of the discussion boards.
Or the interviewers might simply select the most knowledgeable respondent.
So, this is somebody who can generally talk about the entire household or,
and provide family level information.
This is not necessarily somebody who's recruited at random.
Quota methods are another technique that are used in which, for example, for
some households the designated respondent is a female, and for
others the designated respondent is a male.
And then there are the probability or
sometimes called quasi-probability methods that I've already alluded to,
the Kish selection method and the birthday methods, and then there are others.
So turning to the Kish method, just to give you a taste of it,
it's a little more complicated than what we'll cover here.
The interviewer asks the household member to list all of the people living in or
who are members of the household or living in the housing unit and
then indicate which of them is eligible to take part in the survey.
So that might require that they be over 18 years of age, for example.
It varies from study to study.
So once there's a roster, it's called, of the eligible household members,
then the interviewer asks the household member to number or order the eligible
males from oldest to youngest and the eligible females from oldest to youngest.
Then with these really two rosters in hand the interviewer uses what's
called a selection table.
And there are 8 selection tables that randomly allocate household members
to be the respondent.
So this is an example of what one of the selection tables might look like.
Keep in mind that there are others that are randomly assigned to cases or
households.
So the selection table will indicate in a row the number of eligible persons in
the household and then which of them will be selected as the respondent.
So of course if there's just 1 eligible person in the household, that person,
it's person number 1 is selected.
That goes without saying.
If there are 2, it could be person 1 or person 2.
And in this selection table it's person 2.
Another selection table would say in a 2 person household, it's person 1.
If there are 3 people in the household, this selection table points to person 2,
but another table would point to person 1 or person 3.
And if there are 4 or more, this table selects person 3.
Again, other tables would select other people.
And so when these are all combined and equally distributed, every
household member for every household size has an equal probability of selection.
This is the great advantage of the Kish Method that it's the most precise election
method and gives every household member an equal and known probability of selection.
Disadvantages are that it requires gender and age of all eligible household members
to be produced by the household member that the interviewer was speaking with.
And that's time consuming and can be intrusive.
This is that the interviewer's a stranger at this point to the household member.
And so this may be a lot of information that feels personal to give to a stranger.
And it may be subject to interviewer error just because it's complicated.
And interviewers may make mistakes recording this either
in a manual way more likely these days entering it into a computer.
It can be automated and, for example, extended to over sample
groups of particular interest, like teenagers or males, etc.
So the Kish method is still in widespread use despite these drawbacks.
It still has this one advantage that it's a very precise selection method.
The birthday methods are last birthday, next birthday, example wording is,
may I speak to the person of the household who is 18 years of age or older, and
whose birthday was most recent?
That would be the last birthday method.
The next birthday method's exactly the same,
except instead of was most recent would be is next.
And so the interviewer asks to speak to the adult member of the household who has
the next birthday.
So advantages of the birthday methods, they're relatively easy to administer.
They yield an equal probability of selection within a household.
Anyone has the same chance of having had the most recent or the next birthday.
They're much less intrusive than the Kish method, and they're brief.
It's low effort for
the household member in comparison to what happens when the Kish method is used.
And it generates a sample that's more or less random.
Disadvantages are that the exact chance of selection is unknown.
So while there's an equal chance of selection,
we don't know what the actual chance of selection is.
And for this reason, this has been called a quasi-probability method.
It's subject to reporting error because it becomes evident quickly
why the interviewer is asking about birthdays.
And so for a household member who doesn't want to be selected it's easy to indicate
that somebody else had the most recent birthday.
And it's difficult to verify that the report is truthful.
And so it's for these reasons that Biemer and Lyberg recommend against using
the birthday methods in what they call rigorous scientific studies.
As I said there are a number of other methods.
The paper by Gaziano goes through them and
critiques them and nicely compares them.
A comparison of the Kish Method and
the birthday methods was conducted by Binson and her colleagues looking
at dropouts in the two methods in a random digit dial telephone study.
Overall the Kish method performed worse, that is led to more drop outs, and
the next birthday method best, with last birthday method in between.
You can see the dropout rates here.
Kish is the highest, and next birthday is lowest.
What was surprising was that much of this effect occurred before the respondent
could know what type of screener questions they would be asked.
In other words they really didn’t know what would follow.
So you can see the percent who dropped out before the end of the introduction, that
is before the interviewer had actually administered the screening process,
was highest for Kish and last birthday, and lower for next birthday.
So, it's as if the Interviewers are telegraphing to the respondents,
to the household members what would follow.
As if they were somehow indicating that the Kish method's going to be quite
burdensome, and as a result,
the household members terminated the interaction at that point.
There's no guarantee this is exactly what happened.
But I mean, the empirical result is that the Kish method did promote dropouts,
and so in that sense, was less successful when it came to recruitment.
And presumably this has something to do with its burden.
Whether this was evident before the method was actually administered or
not is not 100% clear.
But it seems likely from this study.
More recently a hybrid model has been proposed that is
much less invasive than a Kish alone, and
in most cases, can be explicit about the probability of selection.
So there's a compromise here, but the idea is this is sort of the best compromise.
And so the way it works is that the interviewer asks the respondent
in a screener interview, that is before the actual interview begins,
how many adults are in the household?
If there's just one, if it's a one adult household, then that person,
the screener respondent is selected, and that's the end of the selection process.
If more than one adult was in the household,
then a random selection process is administered.
So that if there are two household members, and the screener respondent is
not selected, that is the person providing answers to the questions like,
how many people live in this household?
If that person is not selected,
then the other person is selected as sort of a coin toss.
What this means is that there's no need to
engage in a selection method like the Kish method.
If there are more than two people and the screener respondent,
the person the interviewer's talking to, is not selected, then and and
only then, this eliminates a lot of situations.
Only then is the last birthday method used.
In very rare cases where birthdays are unknown, the Kish method is used.
So it really reduces the number of situations, the number of cases,
the number of occurrences of using any selection method,
but particularly the Kish method.
So this was tested, well developed, and tested by Rizzo et al.,
in a study of where Americans acquire their health information.
As a kind of first check their work, the proportions of males and females who were
selected were close to what is expected from the current population survey.
So that's an indication that this is working well.
As an indication that this really did reduce the amount of what they called
additional sampling, that is using the birthday method or the Kish method, you
can see that when there was 1 person, of course, no additional sampling is needed.
Where there are 2 people, no additional sampling is needed.
It's basically a coin toss.
When there are 3 people, adults in the household, additional sampling is needed
two-thirds of the time, that is when the screener respondent is not selected.
And it continues like that.
When there are 4 people,
additional sampling is needed three-quarters of the time.
When there are 5 or more, additional sampling is needed 85% of the time.
But this substantially cuts down the number of households that need
additional sampling when you look overall to 12.2%.
So that's a great savings when you look at the impact of this procedure
on break-offs or dropouts, for example.
So choosing among the alternative selection methods,
they're all subject to response error.
So the Kish method requires the household member to enumerate who lives in,
all members of the household.
And that may not be entirely truthful.
The birthday methods make it evident why the question is being asked in
the first place.
And this is something that a household member can manipulate if he or
she doesn't want to participate.
Full listings of household require that the interviewer explain the rationale
to the respondent, or
to the household member who's being asked to provide this information.
And that may affect response rate in some cases.
As we saw there was more dropout in Kish than in birthday method
which doesn't require a full enumeration.
Finally there are different organizational cultures regarding preferences for
the different methods.
So some survey shops just use one of these or predominantly use one of these.
And then different clients have different preferences.
So a survey organization will change the selection method depending
on what the client believes to be the most appropriate.
We'll talk next about a related topic known as proxy responding.
And the issue there is what if the respondent,
the household member who has been selected as a respondent isn't available?
Is it acceptable to have another household member, usually the household
member who is engaging in the selection process serve as a proxy,
a surrogate, for the household member who has been selected?
This differs from survey to survey, organization to organization.
But it is done, and it can save costs by
making it unnecessary for an additional phone call or an additional
visit to a household to be made if the interview can be conducted then and there.
So we'll turn to proxy responding next and
take a look at what the implications are for response quality.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
