You pick up your iPhone while waiting in line at a coffee shop. You google a not-so-famous actor, get linked to a Wikipedia entry listing his recent movies and popular YouTube clips of several of them. You check out user reviews on Amazon and pick one, download that movie on BitTorrent or stream that in Netflix. But suddenly the WiFi logo on your phone is gone and you're on 3G. Video quality starts to degrade, but you don't know if it's the server getting crowded or the Internet is congested somewhere. In any case, it costs you $10 per Gigabyte, and you decide to stop watching the movie, and instead multitask between sending tweets and calling your friend on Skype, while songs stream from iCloud to your phone. You're happy with the call quality, but get a little irritated when you see there're no new followers on Twitter. You may wonder how they all kind of work, and why sometimes they don't. Take a look at the list of 20 questions below. Each question is selected not just for its relevance to our daily lives, but also for the core concepts in the field of networking illustrated by its answers. This course is about formulating and answering these 20 questions. All the features of this course are available for free. It does not offer a certificate upon completion.
The Wisdom of Crowds
Loading...
Из курса от партнера Princeton University
Networks: Friends, Money, and Bytes
62 оценки
Из урока
When Can I Trust an Average Rating on Amazon?
The decision of whether or not to make an online purchase is often driven by feedback that has been left by past customers, commonly in the form of star ratings. In this lecture, we will study Amazon's review system. In doing so, we will explore some of the methods for, and challenges behind, rating aggregation.
Познакомьтесь с преподавателями
Mung ChiangProfessor
Electrical Engineering
Christopher BrintonLecturer
Electrical Engineering
So the first question we're going to raise now is, how do we understand the
effectiveness of averaging a crowd? Let's take the Galton experiment.
And, let's say that the true weight of the ox is x.
This number to be any positive number and follows certain distribution.
So, we'll soon be talking about the expectation of error relative to this, the
distribution of this x. But everybody got some error term, which could depends on
the actual weight x and also differs from one person to another.
We call this the estimate y sub I for the i-th participant in the experiment as a
function of x So this is an additive error model.
Of the estimate provided by each person, and let's say there are n of these people.
So, i goes from one, to da, da, da, to n say 787.
Now we're going to assume that these error terms are both unbiased and independent.
Unbiased means that the expectation of this error term for each user I as a
function of x, Taken over the distribution of. x is zero.
So, sometimes you may overestimate, sometimes you underestimate but, you would
not systematically over or underestimate this goes for all users on.
That's unbiased independent means that this error term only depends on i, it does
not depend on any other user j. In reality neither unbiased nor
independent assumption is true in general. We'll often see biased, systematic bias,
certain people tend to overestimate while others tend to underestimate and sometimes
these errors also dependent. In fact, in lecture seven we'll look at
example of information cascade, where. The dependents of estimates destroys the
wisdom of Kraus. Now, for Amazon review, actually.
Is it unbiased or independent? Essentially not unbiased.
[inaudible]. Although you may be able to use user ID
and history of her rating to do some normalization.
And it's sort of independent. Most people enter review based on her own
opinion. So even if she can see the existing rating
and reviews, she may not be reacting in response to them.
But sometimes they do. Sometimes a review says that, the previous
reviews are certainly biased. I'm here to correct that.
So there is also some element of dependence on Amazon.
Alright, so now we are going to look at the so called wisdom of crowds.
Exemplified by Garden's experiment by comparing two terms, one is the average of
errors made by each individual participant, the other is the error of the
averaged estimate, and the hope is that the error of average estimate is much
smaller than compared to the average of the error.
Let's first look at average of error this is easy, the error term Claire is epsilon.
I of participant i and we're going to use our two norm or mean squared error.
Just like last lecture for Netflix, except over there we use the Ruben square.
Sometimes we take the square root but the idea is the same.
I'm going to use the squared error as the metric to quantify error terms.
So, we'll look at this thing squared and then we'll look at the expectation of this
term over the distribution of x and then we'll sum, sum over all the participants
from one to n. Take the average by dividing over n, and
this is what we call the, error term for the average error.
Okay, or averaged square to be more precise.
So just remember this expression. This is the averaged squared error.
On the other hand, we also want to look at the error of the average.
First of all, what is the average? Well, average is the sum of Yis divided by
n. And the error of the average is that minus x,
Okay? This could be positive, it could be
negative but this is the same as one over n times sum of Yi minus n times x,
obviously. So we can also write this as the sum of
one over n times summation of Yi - x by bringing x inside the summation here cuz
there's effective n, we're okay. And this is just one times the summation
of errors. Cuz each term inside the summation of y - x is just epsilon i by
definition. So now, this error of the average is what
we want to look at. And we're going to look at this error of
average as the expectation over distribution of x, of one over n summation
epsilon i. This whole thing squared which equals one
over n squared can take that out of the expectation times the expectation of the
summation epsilon Ix. Summing over i. This whole term squared.
Now you say, look this expression isn't it the same as the last expression.
Looks like saying there's and summation expectation epsilon squared but not quite.
This is taking the sum of the expectation of squared epsilon.
Whereas this one is the expectation of this square of the sum.
So, this one is talking about square of summation, not the summation of squares,
and the two are clearly different. For example, this two user case.
Epsilon plus epsilon two squared, which is what this is about, is epsilon squared
plus epsilon two squared plus two times epsilon one, epsilon two.
This, is what, the averaged error is about.
This is the error, of the average. The difference is the collection of these
cross terms. But, if they are independent,
Then we say that epsilon times epsilon two, You take the expectation over
distribution x on this expression is going to be zero, because these two error terms
are not correlated with each other. So, these cross terms are cancelled.
And in that case, sum of square, and square of sums are the same.
And therefore, this expression is the same as this expression.
So, the only difference is the multiplication factor in front of it.
In this case its one over n. In this case its one over n squared and therefore we
have the desired relationship, that is the error term of the averaged, the average
first. Then you look at the error, is actually only one over n of the error
term, if you look at the averaged error. So compute error first of the individual
ones, and then take the average versus average first and then look at the error
term. These two are different bi effector,
multiplicative effector of n. And this multiplicative effect of N is
codified as an example of wisdom of crowds in Galton's experiment.
This as simple averaging can work if this unbiased and independent estimates.
There's no bias terms, and these cross terms cancel each other under the
expectation of distribution backs. And we have effected n enhancement.
If there are five people, is five times better.
If there are 1000 people, it's 1000 times better in terms of the mean square error.
Now what if they're completely dependent? Now we don't need these calculations at
all. Whether you air it first in the typical
error of that wisdom of crowns, or you just look at the average of the individual
errors doesn't matter. They are the same.
If they're completely dependent. If they're somewhat dependent, then you
would have to look at the expectation of these error terms correlation in more
detail. But you'll be somewhere between the vector
one versus a vector of n or one over n, depending on how you look at it.
This sounds very promising encouraging with a simple duralation, which seems to
be able to identify the root cause of the wisdom of crowds.
Well, and let's look at the positive side before we look at some cautionary remarks.
The positive side says that as long as they're dependent of each other, then you
have the wisdom of crowds. This particular type of wisdom has nothing
to do with identifying who's the expert. It just says that you might be wrong in
all kinds of directions. But as long as you are wrong in different
directions, then we will have the power of this fact of n enhancement of their
return. In hindsight it's hardly a surprise, this
is really the law of large number, plus the convexity of convex quadratic function
at play here. Now, you may object first.
What if I actually know who are the experts here, maybe she's a farmer.
Well in the advanced material, we'll look at how we can use boosting methods to
extract the more important opinion by the experts.
There will be a very different philosophy. This philosophy in what we just talked
about, only depends on the fact that you can all be wrong as long as you're wrong
in different ways, independent of each other.
Now second objection is what about the scale?
This factor of n effect holds even when n is two or when n is 1,000,000.
To most people the wisdom of crowds means that there should be some threshold value
and star above which you see the effect below, which you don't.
But this one applies just as well to a two people case, so that's some mystery that
remains to be needs to be resolved. The third objection or to say the addition
to this discussion is that this vector n is only one view on [inaudible] crowds we
call the, multiplexing game. There's another real covert diverse
thinking that says if there's some bad events that you didn't want happen, then,
by putting n of these entities together you get a one minus, one minus P bracket
to the power n whereas n is the size of the crowd.
In fact, this is called the diversity gain.
This is called the multiplexing gain. We will encounter this diversity gain in
later chapters. In fact, we encounter both kinds of gains
in [INAUDIBLE] in both technology and the social networks.
Between wireless networks, on the WiFi LTE network, all the way to Galton's example.
But this is our first encounter on one side of this coin.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
