This course presents research conducted to increase our understanding of how data collection decisions affect survey errors. This is not a “how–to-do-it” course on data collection, but instead reviews the literature on survey design decisions and data quality in order to sensitize learners to how alternative survey designs might impact the data obtained from those surveys. The course reviews a range of survey data collection methods that are both interview-based (face-to-face and telephone) and self-administered (paper questionnaires that are mailed and those that are implemented online, i.e. as web surveys). Mixed mode designs are also covered as well as several hybrid modes for collecting sensitive information e.g., self-administering the sensitive questions in what is otherwise a face-to-face interview. The course also covers newer methods such as mobile web and SMS (text message) interviews, and examines alternative data sources such as social media. It concentrates on the impact these techniques have on the quality of survey data, including error from measurement, nonresponse, and coverage, and assesses the tradeoffs between these error sources when researchers choose a mode or survey design.
3.2.3 Personal vs. Formal Style, I-R Rapport
Loading...
Из курса от партнера University of Michigan
Data Collection: Online, Telephone and Face-to-face
131 оценка
Из урока
Module 3: Interviewers and Interviewing
In this lesson, we explore the various roles interviewers take on beside asking questions and collecting answers, as well as some of the different approaches to interviewing that have been proposed and how they affect the accuracy of responses. By the end of Lesson 3, you will appreciate the benefits and costs of collecting data in interviews and will be able to contrast them with the costs and benefits of self-administration.
Познакомьтесь с преподавателями
Frederick Conrad, Ph.D.Research Professor, Survey Methodology
Institute for Social Research
Welcome back.
Continuing with our discussion of interviewing technique,
we just finished discussing the distinction between standardized and
conversational interviewing.
We're shifting our focus now to another distinction,
namely the distinction between personal and formal interviewing techniques.
Or interpersonally engaged might be another way to think of personal,
versus business-like might be another way to think of formal interviewing.
And the question is, which approach produces higher-quality data?
This distinction really, well I think it had been in the literature for some time,
it really was made clear and studied in depth by Dijkstra, and
later with his colleagues Van der Zouwen and Smit.
They explored the distinction between these two interviewing techniques.
And I should just point out, and
I'll reiterate it, this is really orthogonal to, or independent of,
the distinction between standardized and conversational interviewing.
So while formal might sound like standardized, and
personal might sound like conversational, they're really not analogous.
You can have a formal interview that's not at all scripted.
It's just that the content and
the demeanor of the interview are very business-like.
And you can have a conversational interview that's not
particularly personal,
in which the interviewer doesn't attempt to make a connection with respondents but
does clarify the meaning of questions, so just want to reiterate that.
So Dijkstra looked at this in 1987 and noted that the two
techniques shared a lot, namely that the interviewers read questions and probed.
But it really had to do with whether they attempted to
make a connection with the respondents, or instead,
really retained a kind of professional, business-like orientation.
So for the formal style, what Dijkstra reported was,
this actually required interviewers to kind of unlearn their everyday
interviewing practices, where they did,
just because it was kind of more natural, attempt to connect with the respondents.
It required some training to be more business-like.
The personal style involved providing feedback to
respondents with expressions like, that's nice for you, I have similar feelings.
So you can see, these are quite different.
And you might wonder if the personal style is leading or somehow introduces
the influence of the interviewer in a way that would be undesirable.
And that's really what he wanted to test.
So in his study,
the interviewers are randomly assigned to one style or the other.
And then he behavior coded the interaction between them to better
understand how the two interviewing techniques worked.
So Dijkstra really evaluated the impact of interviewing technique on data
quality in two ways, by looking at accuracy and social desirability.
Really, the question he was after was whether the personal style
led to the greater motivation by respondents to work hard and
provide accurate or as accurate answers as possible.
Versus, did it lead to ingratiation, that is, the tendency to say what they thought
the interviewers would want to hear, rather than what was actually the case.
So the accuracy measure was the proportional correctness of a map of their
hometown that he asked them to draw, or that the interviewers asked them to draw.
Was it proportionately correct and did they include the right landmarks,
among other attributes?
And the idea was that if they were motivated to try hard and
provide high-quality data, they would provide more accurate maps.
And the measure of social desirability was their score on a reduced version of
the Marlowe-Crowne impression management scale,
which is a standardized questionnaire that operationalizes impression
management in a series of items.
And the idea is that if they scored higher on the Marlowe-Crowne instrument,
then they were more prone to provide socially desirable answers.
So what he found was, if you look first at the accuracy scores,
in which higher scores in this top table represent greater accuracy, so
we look across the top row and compare it to the bottom row,
the numbers are bigger for personal versus formal interviewing.
So the personal interviewing style is leading to more accurate maps than
the formal style, which is what the prediction had been.
The concern was that this could be the case, but
it would lead to greater social desirability by increasing ingratiation.
So if you look at the lower table, you can see that actually,
this concern was not supported.
The personal style, if anything,
led to lower scores than the formal style for both males and females.
So it appears that accuracy does not come at the cost of increased
social desirability.
So Dijkstra followed up on this initial study with his
colleagues Van der Zouwen and Smit.
And what they did, among other things,
was to actually ask questions that had more and less socially desirable answers,
rather than just administering an impression management scale.
So a more direct measure of whether the personal interviewing style would
increase ingratiation, would lead to more socially desirable answers.
The questions they asked has come to be regarded as moderately
socially desirable or moderately socially undesirable.
So for example, they asked the question, after moving to this neighborhood,
did you try to make contact with people living here?
The desirable answer would be to say, yes, I did, even if that's not the case.
But it's not so stigmatized to say no, to give the undesirable,
accurate answer to that.
One could imagine respondents would be willing to do this, at
the same time that they felt they'd made a personal connection with the interviewer.
We'll look in a bit at a study which asked more sensitive questions than this, and
examined whether the relationship with the interviewer
affected the social desirability of responses.
But for this question, if you look across the table, you can see that about 18%
more respondents answered no, gave the undesirable answer, the more truthful
answer presumably, when interviewed in the personal style than in the formal style.
So that would suggest that they are giving more truthful answers,
they're not ingratiating themselves to the interviewer,
when the interviewer has made a personal connection with the respondents.
A second question with a moderately socially undesirable answer is,
are there people living in this neighborhood who you do not like?
Well, there's no difference in the number of respondents, the percent of
respondents, who gave the undesirable and presumably more truthful answer.
But 5% more said, don't know, when interviewed in the formal style,
suggesting that at least some of them were avoiding giving the truthful answer,
namely, yes.
So it looks like there is some evidence that when interviewers make a personal
connection, essentially establish rapport with a respondent,
that this can not only increase the accuracy of their answers, which is
perhaps the most important aspect of interviewing, but will increase
accuracy without increasing socially desirable answers, so called ingratiation.
Now the mechanism really does seem to be establishing rapport that
leads personal interviewing to improve quality in these ways.
So there actually is a literature about the rapport between interviewers and
respondents, which for a long time was relatively inconclusive.
But I'd just like to go over some of the findings, because they sort of set
the stage for some more recent work that's a little bit more definitive.
But early on, Sheatsley defined rapport as essentially demographic similarity.
That is, people would feel a sense of closeness and
openness with other people who had more of the attributes that they had.
So it's kind of the opposite of social distance, a sort of social closeness,
social proximity.
This is just one interpretation, or one way to operationalize it,
but this is what Sheatsley proposed.
The idea was that this would lead to more honest and thoughtful survey responding.
But as I said, the literature has been kind of inconclusive,
at least from these early years.
So Weiss found, for example, lower validity comparing answers to
administrative records when rapport was higher,
suggesting that rapport actually harms the accuracy of data.
But Goudy and Potter found essentially no relationship between rapport and
the interviewer's performance,
suggesting that rapport might really be kind of a red herring.
They measured this on a several-item scale administered to the interviewer.
For example, during the interview, how ill at ease did the respondent feel?
Well, with hindsight, we can recognize this is probably not the best way to get
at the respondent's sense of the rapport they had established with the interviewer.
So this was really the state of the literature for a long time, and
people kind of abandoned this line of work.
One kind of unanswered question is whether, in fact, rapport may be due to
the interviewer's behavior if you measure the respondent's judgement of rapport,
for example, the interviewer's nonverbal behaviors,
maybe more than their fixed attributes, the social proximity idea.
And it may also be that the level of sensitivity of the questions
may moderate the effect of rapport, so that for moderately sensitive questions,
increased rapport may increase disclosure.
But for highly sensitive questions, it may reverse and lead people to be less honest.
Because now they actually care about their relationship with the interviewer.
They've established rapport, and they care about what the interviewer thinks of them.
So to report behavior that stigmatizes them may be something they don't
want to do.
So rapport could cut both ways depending on the level of sensitivity.
So turning first to whether the idea that rapport has to do with fixed attributes
is correct or not, or maybe it's nonverbal behavior, or maybe it's both.
A study by Johnson and his colleagues suggested that
demographic similarity really does increase disclosure.
Now whether it's exactly because rapport has been increased or
not is not evident in this study.
But it's a rather startling finding.
What Johnson did was to operationalize social distance
as the overlap of demographic features between the interviewer and respondent,
much as Sheatsley had proposed years before,
where more overlap equals less social distance and greater proximity.
The attributes that the authors measured and
examined relationship of to disclosure were age,
gender, race and ethnicity, and education.
And so the question was, how does interviewer distance,
measured by the overlap in those demographic variables,
affect the disclosure of undesirable behaviors?
The research was done on the telephone, which is why it's a fairly startling
finding that even attributes as
opaque to a telephone conversation as education had an impact on disclosure.
So if you look at the table, the amount of disclosure,
which is the amount of either lifetime drug use or recent drug use, where bigger
numbers indicate more drug use, increases from no overlap to complete overlap.
It increases monotonically for both measures, lifetime drug use and
recent drug use, suggesting that this idea of rapport, or at least something that
promotes disclosure, is in fact related to social distance or social closeness.
But it turns out that nonverbal behavior by the interviewer seems to be
related to rapport as well.
Foucault analyzed the facial behavior of four female
interviewers in the National Survey of Family Growth and
then obtained the respondents' ratings of rapport with the interviewer.
The ratings were based on agreement that particular adjectives
characterized the interview situation, so relaxed, cooperative, unfriendly.
So they presented these adjectives and asked respondents to agree or
disagree that these characterized the interview, both the interview situation
and the interviewer herself, so for example, cooperative, friendly, cold.
This allowed them to compute a rapport measure.
And then they looked at its relationship to different nonverbal
behaviors by the interviewer, which they coded in great detail.
And what they found was that the higher the respondents rated rapport,
the more the interviewers smiled and nodded and the less the interviewer looked
directly at the respondent, so the less direct gaze by the interviewer.
They had no direct measure of data quality, so did increased rapport,
which was related to these nonverbal behaviors, increase data quality?
But in a follow up report, Foucault and
Sun found that the answers were more acceptable when rapport was greater.
And by acceptable,
they meant that they conformed to the format that the question required.
There was no need for the interviewers to ask for additional information.
So we see on the one hand that demographic similarity seems to increase disclosure,
not exactly sure if this involves rapport.
We see that the interviewer's nonverbal behaviors affect respondents'
ratings of rapport, and maybe have some impact on the quality of the answers.
At least, the respondents seem to try
harder to give the kind of answer that they were being asked about.
But as I said before, it's possible that rapport increases
the accuracy of answers to sensitive
questions if they're moderately sensitive, but not highly sensitive.
And so this was really the issue that Sun looked into,
namely to see if the level of sensitivity of the questions interacted with rapport.
That is, increased rapport when the questions went from not very sensitive to
moderately sensitive, and then increased disclosure when the questions went from
not very sensitive to moderately sensitive, and then reduced disclosure
when the questions went from moderately sensitive to very sensitive.
So she obtained ratings of questions in, actually,
a separate study as either high, moderate, or low in sensitivity.
And these were asked by interviewers in face-to-face interviews, and
actually also in video-mediated interviews, but whether they were video or
face-to-face had no impact on this part of the analysis, and
collected respondents' ratings of rapport immediately after the interview.
And what she found was that high rapport increased disclosure of sensitive
information, both moderately and highly sensitive information,
which was contrary to the hypothesis that there might be a reversal for
the highly sensitive questions.
But increased rapport also lead to more item non-response when the questions
were highly sensitive.
That is, respondents in some cases chose just not to answer.
So it seems as though rapport is backfiring somewhat.
So when respondents answer,
they seem willing to give truthful answers to highly sensitive questions.
But in higher numbers, they are less willing to answer these very sensitive
questions when rapport is high, consistent with the hypothesis that as you develop
a relationship with the interviewer, there's more reason to
sort of shade the truth and not answer with complete candor.
Okay, so to summarize what we've been discussing, the personal
versus formal style of interviewing seems to be a real distinction.
The personal style motivates respondents to give accurate and
candid responses relative to the formal style.
So it does not seem to increase ingratiation.
That is, respondents don't seem to be more likely to answer these moderately
sensitive questions in ways that would present them in a better light to
the interviewer in the personal style than in the formal style,
which was the concern going into it.
In fact, if anything, the personal style leads to reports of more
socially undesirable behavior so, at least of this moderately sensitive variety.
The personal interviewing technique seems to work by increasing
rapport between interviewers and respondents.
Again, I want to reiterate, this is not the same distinction,
the personal versus formal interviewing distinction,
as conversational versus standardized, where rapport really isn't relevant.
And in the research on rapport, increased rapport seems to increase disclosure of
moderately and extremely sensitive behavior, but also increases non-response
for extremely sensitive items, as we just saw in the research by Sun.
Finally, the study by Johnson and colleagues is consistent with the idea
that less social distance between the interviewer and
the respondent seems to increase disclosure.
I think the take-home message is that interviewers really can add value,
depending on the technique that they're using.
So they can produce more accurate and
more candid responses with some techniques than others,
but there are trade-offs across the board.
At the same time, interviewers can add error, through what
are called interviewer effects, which is what we'll turn to in the next topic.
Interviewer effects seem to come in two varieties.
Those that have to do with interviewer behavior, really, the extent to which they
are administering the questionnaire in the same way for all respondents.
And the other type of interviewer effect has to do with the interviewer's
fixed attributes, such as race, gender, age, and
the extent to which the questions they ask are related to those attributes.
So we'll turn first to interviewer effects having to do with their behavior.
And really, this focuses very much on the origins of standardized interviewing and
the attempt to really restrict the way interviewers
are able to administer the questionnaire.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
