There have been about five studies that have examined
whether it's worth it to blind or mask the author doing
the systematic review to who the authors are of the individual study.
Their institutions, what journal their article's published in and their findings.
So for example, let's say I'm extracting data for
my systematic review from a series of ten different studies.
If I can see where each study was published,
am I more likely to think that a study that comes from a high-ranking journal
is of higher quality than a study that comes from a low-ranking journal?
If a study was published by one of my close colleagues, but
it's really not a very good study, am I likely to give it a higher mark because
this is one of my close colleagues, and I know that he or she does very good work,
and so it must just have been a fluke that this got out somehow.
So perhaps I'm influenced, knowing who did the study, what journal
it was published in, what institution they come from, and their findings.
Maybe that could influence me.
So, it turns out that five studies have looked at whether you can mask
the person doing the data abstraction for the systematic review to these elements,
and to see whether that makes a difference in the data that they extract.
So in the old days, what we used to do is something called differential
photocopying, and it took forever.
You cut up an article and
you just present to the data extractor the title and the method section.
And then the person would extract the data on risk of bias and
what are called quality items for that study.
Then we would compare the results that were extracted for that study,
that was differentially photocopied, with results of a study extracted where there
was no differential photocopying, and see if they got different answers.
It turns out, really, all this blinding or masking of reviewers to who did
the study and what they found really didn't make a difference.
Although one study in 1996 did find that it made a difference to do masking,
the rest of the studies really didn't so much.
Maybe in specific cases it appeared to make a difference, but
over all, a systematic review and meta-analysis of this
particular topic showed that it doesn't make any difference to mask the authors to
the author's institution journal and results before they abstract the data.
This is good news for all of us because that differential photocopying took
a long time and was a real hassle to do.
So what this means is that you can abstract
data from your individual studies without anything fancy.
Just make sure two people do it, and then you can compare
what the two people extract, and that's a good way to see whether they're finding
the same results and to discuss any differences that they're finding.