Now, suppose you're on Amazon, and you're looking at purchasing an HDTV. And on the basis of price, you've narrowed it down to two different choices. The Toshiba, shown here, and the Panasonic, shown here. So, at a first glance, if you're trying to compare these two, you may first look down at the number of stars that each has received. Right? So, we see that Toshiba has received a four out of five stars. And the Panasonic has received 4.5 out of 5 stars, four and a half stars. So, from a first glance you might say, well the Panasonic is better so I'm going to buy the Panasonic. But let's take a step back for a minute and look at how this average rating is really computed and what goes into it. So, to do that, let's look at what the reviews themselves consist of. So, when a person, enters the review for a product, it contains three fields. The first is a rating, from one to five stars. So, you could choose one star, two star, three star, four star, or five star. The second thing is a textual review that comes along with it which explains why that particular rating was given. And Amazon has gotten very strict about making sure that the textual reviews are meaningful and that they correspond to the ratings well. The the third thing is the number of people who found this review helpful, right? So, you, you can look at any review, and it'll say something like, 30 out of 35 people found this review helpful. So, you can either thumbs up or thumbs down the review. If you're a person, who's looking at the product. So, this average customer review that we see, this 4 and, this 4 this 4.5 stars, is, computed, by finding the means of all the ratings that were entered. So, you take all of these rating values, one to five stars. You add them up and you divide by the total number of reviews and that's how this average customer review is computed. And what this really is is it's an attempt to summarize the opinions of, of a lot of people. Right, so if you're one person, you're not going to have time to read through, for instance, all 95 of these different reviews to make an opinion about the product. So, it's very desirable to just have a single number that you can look at there and say, all this represents the product, right? But at the same time, we need to be a little cautious here, in terms of choice, because really what we need to consider also is how many people, the number of people that this is based off of. So, whether it's 10 or 100 or 1,000 people, and also the reputation of those people, so the reputation of the reviewers. So, reputation is not a thing. So, this is this is a quality and this is a quantity thing. We need to, we need to consider both of these. So, now, if we look back at the two HDTVs again, we see that the one on the left, the Toshiba, was four out of five stars, but is was based on 95 different ratings. Whereas the Panasonic, on the right, was, 4.5 stars, but it was, only based on 8, 8 people. So, the one has a higher number of ratings, and, but a less average. A lower average, the other one is the other way around. And, so intuitively, as we have more people that enter ratings, so as the quantity gets higher, we're going to trust that the average is more meaningful or more valid or more trustworthy, is really the word here. We're going to trust it more. And the reason for that is it's less susceptible to harsh, lenient, and spam reviews. So for instance to take a simple example. Suppose we have a product right now that's received 1 3-star review. So if it just has 1 3-star review, then the average is going to be 3 stars. Right. That's the average right now. Now, suppose somebody comes along and enters a five star rating. Now it's a very high five star. Maybe you're the person who, whose product it is and you want it, you want to promote it, so you enter a five star rating. So, if you say 3 plus 5, over 2, which is the new mean, that's going to be 4. Now, that's it, that's a pretty big increase, so it went up from 3 to 4 just by 1 person entering a rating. Now, on the other hand suppose that this instead of it, this 3 star rating thing based upon 3 people. or, instead of this three-star rating being based, being based on one person, suppose it was based on 101 people. So, we have a three-star review. Three stars which are entered by 100 people. So, now the average is still three, right? But, now if somebody comes in and enters a five, the new average is not going to change very much at all, right? So, if we compute the new average, we have to add up the total number of stars. And then divide by, the total number of entries right? So we have 101 instances of a three star review and we have one single five star review and then we just divide that by 102, right? Because there's a 101 plus 1, 102. And we're going to get, 3.02. Which, really no different, to one decimal place the same as three, so. That person who came and entered a five star rating didn't really have much of an effect at all, alright. So, that's why we say it's less susceptible to harsh and lenient reviews. And some people, the difference between, you know, spam review or harsh lenient review. Harsh reviewers would be people who tend to enter really low ratings for a lot of products. So, if you took, if you looked at them, and you looked at all the products they ever rated, it would tend to be lower than a lot of other people. Maybe they're more critical. Lenient reviewers, on the other hand, would tend to enter, very high reviews. So they'd, you know, they'd enter all five for everything. This is the greatest product ever, you know? This doesn't really give you much information if they always enter the same review. And, as we said also, spam reviewers would be, for instance if it's your product and you wanted to make it seem like it was much better, so more people bought it. Or, if you were a competitor to a product, you would maybe want to give a lot of really low star ratings just to try to downgrade your competition. So now, if we look back again in our example up here, we see that the, the Toshiba was, did have a lower average rating but again it was based on 95 review. So instead of just so quickly choosing the Panasonic we may very well now choose the Toshiba instead. Because for something like a TV we may want more than eight people to have rated it. And 4 and 4.5 aren't that far apart in reason. Continuing on a point of harsh and lenient reviews. Note that we can't just throw out all five star ratings, or throw out all one star ratings. Because some of them are going to be very accurate. For instance, Amazon has this thing called most helpful reviews. The most helpful favorable, and the most helpful critical review. And, you can see that a lot of people find each one helpful. So clearly, some people are, are gathering some valuable information from each of those reviews. In this case right here, we have one that's a four star review. 104 of 106 found it helpful. And in this case, it's a one star review. So it's a very bipolar reaction to the product. but, a lot of people are finding each one helpful. So we, we can't just throw those out, we have to still factor them in. So, all these beg the question of can how we enhance the reliability of average reviews? And in general, we can identify two approaches to this. The first one is to enforce quality. Right. So, to do that we want to screen out all of these quote, unquote bad ratings. we, if we, if we see a spam rating we, we don't want to include it. If we see something that we know is harsh, or harsher than normal, maybe we want to take it out, but we can't always do that per say. So it's a little ambiguous as to how exactly to do that. But one way, for instance, is to require people to actually submit, their name or to only be able to submit a review once, right? So, not be able to rate anonymously, right? So, if you're, if you're entering some spam review or if you're trying to sabotage a product, you wouldn't be able to do that as easily if you can't if you have to enter your name and you have to indicate who you are. And more generally, we have to check the mechanism that's used to enter the reviews, right? So. Who's able to review? Do you have to buy the product first, before you can review? How many times can you review? You know, all these questions need to be answered in order to enforce quality. Another one, too, is what is the scale of the review. And an interesting point of this. Studies have been done which have shown that depending upon the, the scale whether it be one to three one to five star, one to ten or so forth. They illicit different psychological responses in the users. So, they may actually come up with averages relative to the maximum that are different. And to, to kind of give insight in to why that is, right, if, if it's, if the scale, which is one, two or three, right, if you're trying to do a mapping from this scale to, one through five, for instance, you may say okay, well one is the lowest therefore that's going to map down to this one here which will also be the lowest. And three is the highest, which is going to map to five here which is the highest. Then that means that two could either be a two, three, or a four on this scale. And it's, you know, it's not clear, so. It's going to you know, you could have different results depending upon what the scale is. The second one is to consider the review population size. Which is going to be something that we're going to consider quantitatively throughout this lecture. As we saw before, the question is, how large is large enough? So, how large does the population size have to be? Does it have to be 50 people, 100 people, 1,000 people in order for us to be able to trust that average rating? So, the main take-away here for these two things is that there's no clear answers to them. Right, they're going to depend upon a number a factors like the type of product being reviewed. For instance, a movie's going to be very subjective. There are going to be some people who hate a movie and other people who love it. An electronic good is probably going to be much less subjective whereas a im, a restaurant, you know, depending upon the type of food people like, will be somewhere in between. The other thing too is the quality of the review and the reviewer. You could, you could estimate quality of the review by maybe the length of it or some key words that were entered into it. And the reviewer, Amazon, actually has hall of fame reviewers. So, different reviewers have different rankings. But again, it's really hard to quanta, to quantitatively describe what the quality is, and it's, it's not clear. There's no clear answers to that. So, with these challenges, you're likely thinking, well, a plain aggregation is unlikely to work well. Right? But in fact, there's a lot of circumstances, a lot of instances that have been noted in history where opinion aggregation has worked quite well. And we're going to turn our attention to one of those next.
