What makes WiFi faster at home than at a coffee shop? How does Google order its search results from the trillions of webpages on the Internet? Why does Verizon charge $15 for every GB of data we use? Is it really true that we are connected in six social steps or less? These are just a few of the many intriguing questions we can ask about the social and technical networks that form integral parts of our daily lives. This course is about exploring the answers, using a language that anyone can understand. We will focus on fundamental principles like “sharing is hard”, “crowds are wise”, and “network of networks” that have guided the design and sustainability of today’s networks, and summarize the theories behind everything from the social connections we make on platforms like Facebook to the technology upon which these websites run. Unlike other networking courses, the mathematics included here are no more complicated than adding and multiplying numbers. While mathematical details are necessary to fully specify the algorithms and systems we investigate, they are not required to understand the main ideas. We use illustrations, analogies, and anecdotes about networks as pedagogical tools in lieu of detailed equations. Please note that per Princeton University policy, no certificates, credentials or reports are awarded in connection with this course.
Netflix Prize: Logistics
Loading...
Из курса от партнера Princeton University
Networks Illustrated: Principles without Calculus
39 оценки
Из урока
Movie Recommendation on Netflix
One of the perks of having a Netflix subscription is getting recommendations of movies to watch. Behind the scenes, Netflix uses powerful algorithms to determine which will be suggested to each person specifically. In this lesson, we will take a look at the main ideas behind these algorithms.
Познакомьтесь с преподавателями
Christopher BrintonLecturer
Electrical Engineering
Mung ChiangProfessor
Electrical Engineering
So Netflix had an algorithm to do this recommendation scheme called CineMatch,
and CineMatch worked pretty well if you look at its prediction on some sort of a
test set where again you'd hold back some of the values and look to see how it did.
It would be within half of a star of the true prediction 75% of the time.
So 75% of the predictions were within half of a star, which is pretty good.
but Netflix realized, you know, way back in around 2006, how important prediction
was. Prediction's very important for customer
loyalty, to increase user satisfaction, things along those sorts.
Also for inventory control be able to tell which movies are going to be going
out, if you can make a prediction as to what people are going to like and so on,
there's other factors as well. But prediction's is very important just,
if for nothing else than to make sure users are happy with what movies are
recommended to them. So in 2006 they launched what's called
the Netflix Prize, which you may have heard of before.
And it was a Netflix competition. Basically they said that they would
reward $1 million to a team that could improve the RMSE over CineMatch's RMSE
value by 10%. So you have to do 10% better The
CineMatch, and then we'll give you $1 million.
And this is an open, online, and international contest, competition.
Everybody can enter, anyone who wants to form a team could enter, and it really
spurred a lot of research in the community of data mining and machine
learning. We won't look into what those terms
means, per se, but there was a lot of, just a lot of research into looking at
data, trying to deal with it and make predictions on data.
And, so, what this 10% means is that whatever CineMatch gets right, whatever
prediction value CineMatch has, whatever error it is, you have to do 10% better.
So, take that value and multiply it by 0.9, and then you'd get the value that
you had to get down to. So, now let's look at some of the
logistics behind the NetFlix competition, and then we'll look at the timeline in a
minute. they released over a hundred million
ratings to the public, okay, and this consisted of about 480,00 users and
17,770 movies, which is huge. So you think of the tables we showed
before of users, and then you have movies, and then all your values, your
ratings, your five, four or whatever, and so on.
And so, there were 480,000 users, and 17,770 movies released in this data set.
The thing is that this data is really sparse, which makes it difficult.
We looked at what sparse means previously, but really, what it means is
that many users are going to rate only a few movies, right?
So there's, for any given user, he may only rate just a really, really, really
small fraction of this 17,770. I mean rating that many movies is a lot.
but then you're probably wondering, well how is it that then you have 100 million
ratings if users are only rating very few number of movies?
Well, really because a few users actually rate a ton of movies.
Like, some of them rate, you know, even tens of thousands of a scene of this data
set, which is really, really remarkable to rate that many movies.
And so, Netflix released four data sets total, well they didn't release, but
Netflix had four data sets total they were keeping in the bank in this
competition. And so, they they divided it like this.
This is a good way to visualize how it goes.
one of the sets, the set over here was called the training set.
Okay, and that, that set was released to the public, okay, and this is in the
public. and there were 100 million entries in
that set, that they released to the public.
And this is the set that they said, okay, train your algorithms using this, right.
So, this is what you're going to use to basically, set all your parameters right,
to try to get that error down as low as it can go.
Then they also released what was called a probe set.
And they said use this probe set anytime you want, this is to the public, they
said can use this probe set to test how your training set's doing, right.
So this is basically to say, alright well, now we want to hold back, quote
unquote, hold back some of our entries. So you released this huge bank of data
here, you would say, okay, well I'll keep this probe set, and I won't train on the
probe set, and then I'll test my performance on the probe set after I've
trained my parameters, right? So it's almost like, think about it like
this. You're taking a test, right?
You have to go in and take a test. And this this is basically your study
material for the test, right? And this is a practice exam maybe before
the test, this probe set's a practice exam.
And we'll look at why it's only practice exam.
Now, all right, so now there were private elements of that set too, okay.
And there was, these were not released to the public there was what's called a quiz
set right here and then a test set. Now the quiz set, they allowed the,
people to test on the quiz set, once every day.
So that you could submit your algorithm, and you could test on the quiz set, once
every day and see how, good your algorithm was doing.
And they actually based the progress prizes off of that quiz set.
Then the real test for the million dollars at the end had, was on the test
set. And, you know, that would only happen
once where you would test on the test set, and only one time right at the end
that you would see, okay how well you did.
And so again, we draw that analogy again. Okay.
This training set, this is your study material and you use this to study for
the test. This is homework problems or practice
problems before you go into the test. It gives you a gauge how well you think
you're going to do. Okay?
This is, maybe some practice exam that your instructor's given out, but there's
only a few of them, right. Or you can only take the quiz, you, test
on the quiz set a certain amount of times, and this test is the actual test
when you walk into the exam. So how well you do on these two, the
probe set and the quiz set can give you some idea of how you're going to do when
you walk into the test but you never quite know.
Right, there could be some curve balls but it gives you a pretty good
estimation, right. It's you can feel more confident and the
better you train before you take these probes or quizzes the better you do on
those, then the better you expect you'll do on the test set.
So, now the the training set again. This just to review, it's really the
input data to the algorithms. The probe set is used to gauge
performance, quiz set you can test once per day, but that's held by Netflix so
it's private, and the test set is the end result, the real test at the end.
And that analogy that we just made to in class, so we'll bring back that again
here now, because what this did was this sectioning was actually a really a smart
way of doing it because it prevented people from reverse engineering the test
set. So reverse engineering would mean okay,
well if you gave me the test set then what I would do is I could just set
everything in my algorithm just so that it would basically make the test set
error go to zero. But that's not very useful, because there
could be other test sets that we haven't tested on it, and the algorithm would do
terribly on. And you're just basically setting it for
this guy right here. So that would be this reverse engineering
idea. If you reversed engineered the test set
that would be like getting the test before you went in to take the exam, and
learning how to do all the answers and looking in the back of the book, so when
you go in to take the test you'll get 100%, but if the instructor happened to
change the questions on the exam you may not do so well because you don't know the
material, you didn't actually study with the training set.
That's a very good analogy, and good way of thinking about that.
And so this prevented this whole reverse-engineering idea.
Netflix obviously doesn't want that because if you just reverse-engineer the
test set that's not going to be very useful to them in their entire data set
which they need to really have a robust algorithm that works under a lot of
different sets, not just one specifically.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
