To this point, we've been creating convolutional neural networks that train to recognize images in binary classes. Horses or humans, cats or dogs. They've worked quite well despite having relatively small amounts of data to train on. But we're at a risk of falling into a trap of overconfidence caused by overfitting. Namely, when the dataset is small, we have relatively few examples and as a result, we can have some mistakes in our classification. You've probably heard us use the term overfitting a lot and it's important to understand what that is. Think of it as being very good at spotting something from a limited dataset, but getting confused when you see something that doesn't match your expectations. So for example, imagine that these are the only shoes you've ever seen in your life. Then, you learn that these are shoes and this is what shoes look like. So if I were to show you these, you would recognize them as shoes even if they are different sizes than what you would expect. But if I were to show you this, even though it's a shoe, you would likely not recognize it as such. In that scenario, you have overfit in your understanding of what a shoe looks like. You weren't flexible enough to see this high-heel as a shoe because all of your training and all of your experience in what shoes look like are these hiking boots. Now, this is a common problem in training classifiers, particularly when you have limited data. If you think about it, you would need an infinite dataset to build a perfect classifier, but that might take a little too long to train. So in this lesson, I want to look at some tools that are available to you to make your smaller datasets more effective. We'll start with a simple concept, augmentation. When using convolutional neural networks, we've been passing convolutions over an image in order to learn particular features. Maybe it's the pointy ears for cat, two legs instead of four for human, that kind of thing. Convolutions have been very good at spotting these if they're clear and distinct in the image. But if we could go further, what if for example we could transform the image of the cat so that it could match other pictures of cats where the ears are oriented differently? So if the network was never trained for an image of a cat reclining like this, it may not recognize it. If you don't have the data for a cat reclining, then you could end up in an overfitting situation. But if your images are fed into the training with augmentation such as a rotation, the feature might then be spotted, even if you don't have a cat reclining, your upright cat when rotated, could end up looking the same.
