Introduction to Evaluation and Metrics

Loading...

Из курса от партнера University of Minnesota

Recommender Systems: Evaluation and Metrics

132 оценки

Посмотрите похожие видео

В Coursera собрались лучшие преподаватели мира. Вот несколько рекомендаций лично для вас:

University of Minnesota

Recommender Systems: Evaluation and Metrics

132 оценки

Курс 3 из 5 — Specialization Recommender Systems

In this course you will learn how to evaluate recommender systems. You will gain familiarity with several families of metrics, including ones to measure prediction accuracy, rank accuracy, decision-support, and other factors such as diversity, product coverage, and serendipity. You will learn how different metrics relate to different user goals and business goals. You will also learn how to rigorously conduct offline evaluations (i.e., how to prepare and sample data, and how to aggregate results). And you will learn about online (experimental) evaluation. At the completion of this course you will have the tools you need to compare different recommender system alternatives for a wide variety of uses.

Из урока

Preface

Introduction to Evaluation and Metrics3:26

The Goals of Evaluation10:31

Познакомьтесь с преподавателями

Michael D. Ekstrand
Assistant Professor
Dept. of Computer Science, Boise State University
Joseph A Konstan
Distinguished McKnight Professor and Distinguished University Teaching Professor
Computer Science and Engineering

Welcome to the course on Evaluation and Metrics of Recommender Systems.

In the previous two courses, we've looked at a number of ways of doing

recommendation using basic popularity, segmented statistics, content of items.

Then we've looked at collaborative filtering,

how to use user item interactions in order to produce recommendations.

But one thing we've not yet talked about is, how do we know which of these

approaches is going to work better in a particular situation?

Or if they're going to work at all or we should just pick some random movies for

people to watch.

In this course,

we're going to be learning how to match recommenders to their desired outcomes and

measure their ability to meet the needs in these different kinds of use cases.

And also how to optimize the recommenders behavior.

We've seen a number of parameters such as neighborhood sizes,

how do you pick which one you need?

The tools that we give you in this course will answer that question.

>> The key concepts in this course start with metrics.

We'll take you through some basic metrics for accessing the accuracy of

recommenders, prediction, its ability to help in making correct decisions.

We'll look at rank metrics that evaluate the quality of a top analyst and

then we'll move in to more advance metrics looking at concept such as diversity and

serendipity.

We'll look at the whole concept of evaluation without users

understanding what you can do and the framework for dealing with offline data.

And covering up or

hiding parts of your data to evaluate whether you can predict what you'd hidden.

And we'll look at evaluation with users.

Looking at lab experiments and field experiments,

including the online A/B tests.

As well as some work on surveying users and

log analysis to understand user behavior.

As we put all of that together, we'll get a holistic concept of how we can

evaluate our recommender system and use that evaluation to feed forward and

to tune the system and redesigning it to work better.

>> This course is structured with the first week focusing on basic metrics for

evaluating the accuracy of predictions and recommendations.

For this class, it will be an assignment and a spreadsheet and

there'll be a quiz on the topics.

We'll then spend two weeks on advanced metrics and

offline evaluation structures with programming assignment where you'll

actually implement and run a recommender experiment in LensKit.

If you're taking the honors track we recommend getting started on this

assignment particularly early.

Because, it's fairly computationally expensive to evaluate several algorithms

over several data sets.

So, you'll need to leave some time for the program to run.

We'll also have a quiz again over the topics and

then we will have a week on online evaluation.

Throughout this, we have a running assignment, weeks 2 through 4,

where you design an experiment to evaluate a recommenders' system.

These are going to be peer-graded, you're going to be looking at each other's

experimental designs to assess their effectiveness and appropriateness.

And you can treat this as a warmup for the Capstone project at

the end of this specialization if you're planning to take the entire package.

With that, let's get started.

Ознакомьтесь с нашим каталогом

Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.

Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.

© Coursera Inc., 2019 Все права защищены.

Загрузить из App Store

Загрузить в Google Play