We will introduce methods to perform systematic reviews and meta-analysis of clinical trials. We will cover how to formulate an answerable research question, define inclusion and exclusion criteria, search for the evidence, extract data, assess the risk of bias in clinical trials, and perform a meta-analysis. Upon successfully completing this course, participants will be able to: - Describe the steps in conducting a systematic review - Develop an answerable question using the “Participants Interventions Comparisons Outcomes” (PICO) framework - Describe the process used to collect and extract data from reports of clinical trials - Describe methods to critically assess the risk of bias of clinical trials - Describe and interpret the results of meta-analyses
Lecture 5C: Information Bias
Loading...
Из курса от партнера Johns Hopkins University
Introduction to Systematic Review and Meta-Analysis
929 оценки
Из урока
Minimizing Metabias, Qualitative Synthesis, and Interpreting Results
In this module, we will cover minimizing metabias, selection bias, information bias, how to report transparently, qualitative synthesis, and interpreting results.
Познакомьтесь с преподавателями
Tianjing Li, MD, MHS, PHDAssistant Professor, Epidemiology
Bloomberg School of Public Health
Kay Dickersin, PhDProfessor, Epidemiology
Bloomberg School of Public Health
Hi, again.
This is Kay Dickerson.
Now we're moving into Section C, where we're going to talk about information bias
in doing the systematic review.
So, what are our particular concerns
as regards metabias in doing a systematic review?
I would say, in terms of information bias,
we're worried about the accuracy of our quality assessment or the risk of bias,
the accuracy of the data abstraction that's done from the individual studies
included in the systematic review, and how complete that data abstraction was.
Let's talk first about inclusion bias,
as a subset of information bias, that is, and I already mentioned it,
if we know who the study investigators were on a particular study,
or the outcomes that they assessed or what they found,
does that, it affect what we include in our systematic review?
If I know that the Canadian study of women 40 to 49,
of mammographic screening showed a nulled result for
mammographic screening in terms of breast cancer mortality,
am I more likely to make up inclusion criteria that would ensure
that Canadian study is not included in my systematic review?
That's what we're worried about in terms of information bias,
in terms of whether studies included it or not.
Now, how about if I know the results of a study?
Does that change how I abstract the data?
Let's say, a study was actually included, and I know ahead of time
the individual study that I'm abstracting data from had positive results.
Does that change how I abstract the data?
Am I more likely to think they did a good job?
Am I more likely to look to make sure they examine multiple outcomes?
Or maybe I'm more likely to look if they have negative results.
And how about if I know the study investigators, or outcomes?
Just as before, does it affect if the study gets in?
Does it affect my quality assessment?
There'd been numerous studies on other topics that we
tend to think a study is better done if it has positive results.
Or, we tend to think, there are results on this,
that a study is better done if the author is a man compared to a woman.
And so, these things are known in individual studies to
affect our assessment of the quality of the study, would this also be true for
those people who are working on systematic reviews and meta-analyses?
There have been about five studies that have examined
whether it's worth it to blind or mask the author doing
the systematic review to who the authors are of the individual study.
Their institutions, what journal their article's published in and their findings.
So for example, let's say I'm extracting data for
my systematic review from a series of ten different studies.
If I can see where each study was published,
am I more likely to think that a study that comes from a high-ranking journal
is of higher quality than a study that comes from a low-ranking journal?
If a study was published by one of my close colleagues, but
it's really not a very good study, am I likely to give it a higher mark because
this is one of my close colleagues, and I know that he or she does very good work,
and so it must just have been a fluke that this got out somehow.
So perhaps I'm influenced, knowing who did the study, what journal
it was published in, what institution they come from, and their findings.
Maybe that could influence me.
So, it turns out that five studies have looked at whether you can mask
the person doing the data abstraction for the systematic review to these elements,
and to see whether that makes a difference in the data that they extract.
So in the old days, what we used to do is something called differential
photocopying, and it took forever.
You cut up an article and
you just present to the data extractor the title and the method section.
And then the person would extract the data on risk of bias and
what are called quality items for that study.
Then we would compare the results that were extracted for that study,
that was differentially photocopied, with results of a study extracted where there
was no differential photocopying, and see if they got different answers.
It turns out, really, all this blinding or masking of reviewers to who did
the study and what they found really didn't make a difference.
Although one study in 1996 did find that it made a difference to do masking,
the rest of the studies really didn't so much.
Maybe in specific cases it appeared to make a difference, but
over all, a systematic review and meta-analysis of this
particular topic showed that it doesn't make any difference to mask the authors to
the author's institution journal and results before they abstract the data.
This is good news for all of us because that differential photocopying took
a long time and was a real hassle to do.
So what this means is that you can abstract
data from your individual studies without anything fancy.
Just make sure two people do it, and then you can compare
what the two people extract, and that's a good way to see whether they're finding
the same results and to discuss any differences that they're finding.
So what are some other issues that are related to information bias?
Sometimes we're worried about whether the data that we get from a graph
is not accurately abstracted.
You may find, for example, as a matter of fact I know you will find, in some of
the papers that you are looking at for your systematic review, that the only
way that you can get data on an outcome you're interested in is from a graph.
And the graph might be proportions, and
you have to guess then, at what the numerator and denominator are.
Well, that's scary to do, and
a lot of what we do in systematic reviews is scary and involves a judgment call.
But this is one of the scariest, where you're assuming a numerator and
denominator and all you have is that graph.
We do have some software now, I'll show you, that can do this and
apparently is pretty accurate.
You can also go back to the investigator in that email and
say this is what I deduced, looking at your graph.
Is it correct?
And some of you might be able to get an answer.
Another place where information bias might be an issue,
is whether the experience of your abstractor is a factor.
So for example, when we're doing systematic reviews in our Cochrane Eyes
and Vision group, we hire graduate students, and we wonder whether someone
who's brand new at the task isn't as good, isn't as accurate as the person who
has a lot of experience reviewing this type of study and extracting data.
Another question about information bias might be, and
I mentioned it briefly on the previous slide, is it necessary for
two people to abstract the data and then to compare what they found
in order to assure that we have reliable data from that particular study?
I don't think we have complete data on that yet, but
most of us do duplicate abstraction and certainly for the results.
And then finally, the question of course you have by now because
of what I've said in previous sections, can we rely on what's in the publication?
And unfortunately, the answer is probably no.
Whether there's a bias or not is to some extent open to question, but
I think we know they're reporting biases, and so we probably will have to look in
more than one place to see if data agree in those gray sources,
in FDA database, clinicaltrials.gov, as well as the publication.
So what I'm showing you here is an example from JAMA, where there
was a correction made because there was an error in data extraction and analysis.
Now this is probably pretty rare, that someone published a correction and
we're glad for it of course, but errors are made all of the time.
I don't know about you, but I make errors and I am very grateful when there's
another person doing data extraction, because we get tired we read wrong.
It's just human nature, and we're human beings at the end of the day.
So, I think we have to accept that errors are made and
build in some protection for our systematic reviews.
Here's another example I mentioned,
which is how you can extract data from a published article
using a little piece of software, or even just using your ruler.
So, if you're interested in this and you come up against this problem,
there are sources you can use to try to minimize information bias.
And then finally,
the question that I mentioned about whether experience affects accuracy.
These are some data from a 2009 study where people with less
experience were compared with experts, let's call them.
And to see whether they don't do as good a job,
make more errors than the people who have more experience.
What the authors found, interestingly enough, is that the error rates were
similar and it didn't depend on how much experience the person had.
However, inexperienced people took longer than the experts,
but the error rate was the same, and so that's reassuring for us.
Certainly bears replication in another study, but it was initially reassuring,
at least, that the experience does not affect accuracy.
Finally, I've already mentioned a couple of times, that one of the ways we can
protect against errors in data extraction is to have
two people do that data extraction and then compare what they've extracted.
So what are the possible ways that you could extract data for
your systematic review?
The first is you could have one reader go in there and extract the data onto a form.
The second is to have one reader go in, extract the data, and
a second reader come in, look at what the first reader has extracted, and say,
yeah, I agree with it or no I don't.
That's called single data extraction and verification, and
that's what was done in the study that I'm presenting on this slide,
the Buscemi study published in 2006.
And then the third thing that you could do is you could have two separate individuals
go in and extract the data without talking to one another, or
knowing what the other person did.
And that's a completely independent data extraction.
Those two people who extracted independently get together, then, after
their extraction, look at where there are differences in what they extracted, and
they decide between them what is the correct response.
So what Buscemi did is he compared the single extraction plus
verification with double extraction, and he found that there was less inaccuracy,
and overall, a lower error rate with the double data extraction.
It's only one study though and
some people are concerned that this is much more expensive and
time consuming than if you just do a single data extraction with verification.
One of the solutions that was suggested in the IOM Systematic Review Standards was,
perhaps you could do data extraction for
the results that you will be combining in your systematic review.
And for less important data such as the investigators, the journal,
which was published, and
the dates, this could be done by a single extractor plus a verifier.
That was just a suggestion for those with limited resources, and
I think the jury is still out on whether one needs to do double data extraction.
In the systematic reviews that we do in the Cochrane Collaboration,
we do double data extraction, and I hope you will also in this course.
So that concludes the section on information bias.
In the next section we're going to talk about bias in the analysis and
ways that we could possibly prevent or address it.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
