Welcome back. Transfer learning is one of the most important techniques of deep learning and TensorFlow lets you do that with just a handful of lines of codes. Rather than needing to train a neural network from scratch we can need a lot of data and take a long time to train, you can instead download maybe an open-source model that someone else has already trained on a huge dataset maybe for weeks and use those parameters as a starting point to then train your model just a little bit more on perhaps a smaller dataset that you have for a given task, so it is called transfer learning. Transfer learning, and I find it's really cool because we've been looking at convolutional neural networks up to this point, and when we're training with a convolutional neural network we're extracting features from an image and learning those features, learning the ones that match images. But when we download a large models something like an inception that has all of these features that have already been extracted off lots more images than we can use, I find that really exciting because as you mentioned researchers who built these wonderful models of lots of images have been able to share them and today we've been looking at convolutions and convolutional neural networks where from scratch we've been training something off of our data and off of our images to be able to extract features from these and learn them. But now these models that have been trained off with lots and lots of images over a lot of time might have spotted other features that weren't in our dataset and will help us to build better convolutional networks. Yes. There's so much stuff for neural networks to learn about the world. There are edges, corners, round shapes, curvy shapes, blobs and then there are things like eyes, circles, squares, wheels. There's so many things in the world that convolutional neural networks can pick up on but if you have only a 1,000 images or even 25,000 images may not be enough data for content to learn all those things. So by taking an inception network or some other network that someone else's has trained you can basically download all this knowledge into your neural network to give it a huge and much faster start. It's really exciting to be a part of that community where these things are being shared so that people who don't have access to machinery to be able to build massive models like this one, are able to derive the features from the people who have done so. I personally find that really inspirational. Yes. Many days I wake up really grateful to the whole AI community for the openness, so many breakthrough AI ideas are shared in papers on the Internet, on the open source code shared on the Internet, and I think this has been a large reason for the rapid advance of AI. I think to all the learners if you end up doing something consider sharing it on the Internet as well freely to contribute back to this open community that's making all of us go much faster. Definitely. Transfer learning is a technical embodiment that lets us use these ideas to accelerate the whole field. One of the cool things about transfer learning is that it's so simple to implement, in TensorFlow you download a model and then you say set these models as trainable and freeze or lock those other layers and then you just run. Yes. In the course what the students are going to be looking at is the inception model which is a beautiful image classification and classifies a 1,000 images it has been trained on. I think it's like over a million images and what you're going to do is you're going to take a look at one of the lower levels in that. You're going to lock all the way down to that. You're going to add a DNN a deep neural network underneath that, and then you're just going to retrain for those lower levels, and as a result using all of these, I like to call it standing on the shoulders of giants, you're going to be doing all that to be able to make the classifiers that you've been building in the course to date much more efficient and maybe even quicker to be able to reach higher levels of accuracy than if you're training it from scratch. So this week, you'll learn how to implement transfer learning and use it to get your models to not only train faster but also get higher accuracy. Please dive in.
