Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
62,800 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

JS

Apr 4, 2021

Fantastic course and although it guides you through the course (and may feel less challenging to some) it provides all the building blocks for you to latter apply them to your own interesting project.

AM

Oct 8, 2019

I really enjoyed this course. Many details are given here that are crucial to gain experience and tips on things that looks easy at first sight but are important for a faster ML project implementation

Filter by:

51 - 75 of 7,215 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Vinod K

•

Jan 16, 2018

I had taken Andrew Ng's Machine Learning course. I went on to learn Deep learning from other tutorials and I always wished there was a course on Deep learning too by Andrew Ng. And now that there is, It was worth the wait.

1. All the topics are arranged in logical order. So you feel like a tour of deep learning. Earlier I had to refer to multiple sources for different topics and they usually had different naming and notations which were really confusing.

2. Having taken about 6 top rated courses on AI domain, I can assure you Andrew Ng is the best in his teaching style and content.

3. Exercises and theory go hand in hand. So, you know how to implement as soon as you learn theory.

4. Out of a lot of techniques in each topics like Optimization, Regularization etc. this course picks the most contemporary techniques. This helps you not to wonder which techniques to use in your work.

Overall, This Specialization is like a cookbook for AI. My appreciation and gratitude to Andrew Ng and his team for their contribution to AI.

By Shibhikkiran D

•

Jul 7, 2019

First of all, I thank Professor Andrew Ng for offering this high quality "Deep Learning" specialization. This specialization helped me overall to gain a solid fundamentals and strong intuition about building blocks of Neural Networks. I'm looking forward to have a next level course on top of this track. Thanks again, Sir!

I strongly recommend this specialization for anyone who wish get their hands dirty and wants to understand what really happens under the hood of Neural networks with some curiosity.

Some of the key factors that differentiate this specialization from other specialization course:

1. Concepts are laid from ground up (i.e you to got to build models using basic numpy/pandas/python and then all the way up using tensorflow and keras etc)

2. Programming Assignments at end of each week on every course.

3. Reference to influential research papers on each topics and guidance provided to study those articles.

4. Motivation talks from few great leaders and scientist from Deep Learning field/community.

By Weinan L

•

Feb 5, 2018

Used to tune hyper parameters based on experience... after this course, know more about the internals and from now on, not just know HOW to tune, but WHY it needs to tune this way.

As always, Andrew did fantastic work here to help explain complex formulas in simple and CLEAR way.

Highly recommend it to anyone who fight with overfitting, hyper parameters tuning, etc. It may not help you instantly become a better AI person or help you immediately help you on your day to day programming - as you most likely use various frameworks (Keras/TensorFLow/PyTorch) instead of raw NumPy. But it does help you in the long with better knowledge. It is kinda like show you how the engine works, before teach you more driving skills. It won't help you when your car is working fine, but when it breaks, you know how to troubleshoot and what is the right direction to go. Honestly, I personally think the debugging part is the toughest part of AI.

Take it. Period.

By Zeyad O

•

Apr 15, 2020

I'm Zeyad, an undergraduate of Computer Engineering at Alexandria University in Egypt.

Taking this course really helped me to learn and study this field and also to implement it. It helped me advance in my knowledge. This course helped me defining Deep Learning field, understanding how Deep Learning could potentially impact our business and industry to write a thought leadership piece regarding use cases and industry potential of Machine Learning.

This specialization helped me identifying which aspects of Deep Learning field seem most important and relevant to us, apparently they were all important to us. Walking away with a strong foundation in where Deep Learning is going, what it does, and how to prepare for it.

Deep Learning specialization helped me achieving a good learning and knowledge about that field.

Thank you so much for offering such wonderful piece of art.

Best Regards,

Zeyad

By Vaibhav M

•

Aug 9, 2023

Amazing courses that go into detailed explanations about the math and intuitions behind the algorithms without getting too convoluted or making things unnecessarily complicated just for the sake of it.

Prof. Andrew doesn’t just tell you the name of a function for a library (like scikit

learn or tensorflow) and give you magic numbers for parameters. You actually build the model yourself and learn what the parameters stand for and what is the purpose of those parameters and hyper-parameters.

The specialization is well divided into meaningful courses and each course is well structured so that you know exactly what you are going to learn and what key specific skills you will get after completion of a course. Because of the quizzes and practical labs, after completing a course you actually gain confidence that you can design optimized solutions for that particular set of problems.

By Alec T

•

Jul 28, 2022

Outside of an academic space, I think this course is one of the best places to be introduced to deep learning. It think it does a tremendous job of orienting you around issues relating to implemention of neural networks for deep learning. The information if transferred in an efficient and concise manner - the videos are created in a way that are not too long or too dense in any one sitting to be overwhelmed. I do think this course is best supplemented or partnered with a course that requires more implementation and generation of neural networks. The activities can be performed swiftly if you have been paying attention to the lectures, which may leave some wanting slighty more from an application perspective. I can however admit this is just the 2nd course of 5 in the specialization so take this information with a grain of salt.

By Fabricio Q

•

Jul 30, 2023

This course equips you with the perfect tools to get started on the techniques required to optimize and improve your neural networks. The materials are easy to grasp, and the content is well-planned; I highly recommend the course. It provides a comprehensive overview of the subject matter, and Andrew Ng is knowledgeable and engaging.

As a plus, the interviews with the heroes of Deep Learning provide invaluable insight to newcomers to the Deep Learning domain.

The course materials are well-organized and easy to follow, making it easy to stay on top of the material. Additionally, the assignments are challenging but manageable, allowing ample opportunity to practice and apply the concepts learned in class. Overall, the course was a valuable learning experience and I would recommend it to others throughout the course.

By Johan D R P

•

Dec 9, 2020

I found this course pretty useful to understand a large set of options to explore in a Neural Network (and its inputs) in order to improve its performance. It shows the mathemical fundamentals behind the concepts, without going too deep to confuse a person without advanced calculus knowledge, like me.

However, I would like that the following changes were made:

Update last week lab to Tensorflow 2.0. This framework update seems to be more beginner friendly, because its interaction with Python functions (no need for sessions). Also, a lot of things shown in the lab are deprecated.

Make a lab for hyperparameter exploration. While this task can take a relatively long time, maybe it would be feasible to explore hyperparameters over a simple model. I felt that the course needed more hands on in the part of exploration.

By Steve S

•

Aug 20, 2022

Nothing but kudos on this and the pevious course. While already had a certificate in CNN and Google's Tensor Flow platform, I needed to understand the inner workings of forward and backward propagation. Great combination with Andrew NG (as usual), Jypter notebooks, and the development personnel on topic segmentation, progress, and testing. I loved the "gentle" approach to calculus, differential and partial differential equations and linear algebra. While a periodic note would pop up saying that xx% of people taking this course viewed this particular segment twice, I chose to realy view it in detail. This meant producing a chapter (for each segment with the transcript), adding equations from Andrew's drawings, using and pasting in any Excel computations, and highlighting and emphasing important topics.

By Anirudh K

•

May 8, 2020

This is a really informative course and really crucial if you are planning to do a personal project or even prepare for interviews. It equips you with all the tools to get started with actually start implementing Neural Networks for a problem by 1) Teaching how to prepare data sets 2) Regularization/dropout to increase accuracy on test set, 3) Set up your optimization problem 4) Teach different Optimization Algos 5) Teach Hyperparameter tuning and the order of importance of different hyperparameters 6) batch Normalization and lastly Tensorflow. Andrew NG is truly a master in teaching concepts in an approachable and intuitive way. I believe the course can be made even better by adding Keras to the programming frameworks module along with more videos and programming exercises for data pre processing.

By Jacob J

•

Jun 4, 2023

Excellent course tailored to motivate and educate a real practitioner and less for a researcher.

(1) Loved the course, especially, the intuition-focused approach and not going through each of the mathematical derivations, but focusing on the concept behind the use of each mathematical strategy.

(2) The quizzes didn't make it easy and ensured that the concepts were well grasped. Need the ability to save the answers for further review and study.

(3) The programming assignments were of the right level of difficulty. These assignments may feel easy for someone looking to reinvent the wheel by coding from scratch. However, for a practitioner, the assignments reinforce the sequence of steps in one's mind and help focus on programming areas where one is most likely to make mistakes.

By Valery R

•

Aug 17, 2021

The content of the course is great, but I didn't like at all the final programming assignment on TensorFlow. I'm not a fan of all those third party libraries, or software (not sure how this should be called) which add one more layer between the user and the machine. We call functions without really knowing what we are doing. We just do what the assignment tells us to do. That's relatively easy, but it the assignment could be done without understanding the content of the course. All we have to is simply follow the instructions. We end up with a 100% grade, and we almost have no clue about what we did. I prefer much better to program my own implementation of the neural network in C++ (or Python since this is fashionable), which really forces me to understand what I am doing.

By Nigel S

•

Jun 9, 2019

It explains a bunch of complicated maths and methods in a way that is at least comprehensible by mere mortals, though not necessarily easy. Put another way, if this course doesn't enable you to understand how to tune and optimise deep neural networks, then you probably never will.

The content taught in this course is really valuable because it explains a lot of what is going on behind the scenes in the existing Deep Learning Frameworks like Tensorflow, Keras, etc, and enables you to be a lot more competent and confident in producing effective models in a time-efficient way, than if you didn't have this knowledge.

It also seems to have been built by peopel who not only know the material intimately, but who recognise that many of the learners are very time-poor.

By Aakash S

•

May 28, 2021

Just as I started this course, a person building a DNN using Tensorflow and Keras reached out to me for help. He had spent a day struggling to figure out how to improve the performance of his model to improve the accuracy for training and validation (aka dev) dataset, and then figure out why his model was so inaccurate for test set, even as his model was very accurate for the dev set. After going through just the first week lessons of this course, I was able to suggest to him what approaches to take to optmize hyperparameters of his model to get the desired accuracy for all datasets (tarining, dev and test). It seldom happens that one gathers enough insight after just a week of lessons to apply it towards solving real world problems!

By GURAJAPU S S

•

Jul 30, 2021

This was amazing stuff from the team. Very good to get going on Deep Learning. I have some comments on the course curriculum, regarding the optimization techniques(OTs). The theory presented here is good, but as a learner seeing it for the first time, I didn't get a deep intuition on why we are doing a particular OT and why we need different OTs. So I learned this part from the NPTEL course on Deep Learning by Mitesh Khapra, where he first mentions the reason behind developing an OT, and then he deep dives into the implementation of the technique. In a great way, he developed the concepts on Momentum, NAG, Adagrad, RMS Prop, Adam progressively. Probably an extra video on the analogies behind the OTs can be added on coursera:)

By Taylor B

•

Jun 23, 2019

I took the Machine Learning Course from Stanford with Andrew Ng a few years ago and enjoyed it but I was also somewhat overwhelmed by the math. In contrast, this is my second course in the deep learning specialization and I feel like so far the courses have struck a good balance, introducing core concepts and derivations for things but also making sure I get guided practice along the way, and also not moving straight to frameworks but having students code more or less from scratch first. I'll probably need some practice on kaggle or other datasets as well as reference to a few other learning materials to feel like a strong practitioner, but this gives the tools to make that possible and I'm very satisfied with this result.

By Jorge L

•

Feb 17, 2019

All the courses in the Deep Learning Specialization are very good and met my expectations. I was guided through the nitty-gritties of neural networks, fortunately with a strong emphasis on Computer Vision (my area), deep diving in coherent coding exercises. Prof Andrew, as always, managed to connect the points between theory and practice, recollecting the concepts treated in past lectures, while showing how Tensorflow operates and how to use it. If you ask me, I'd say that the slides of the Machine Learning course used to be better than the slides for the 4 courses in this specialization, in the sense of being useful as studying guide for the future. The current slides only make sense to those who went through the course.

By Luca C

•

Jan 27, 2019

Knowing this makes the difference. How do you evolve from being a monkey behind a keyboard knowing how to tensorflow a NN to homo sapiens? The concepts provided in this course will make the job.

pros: + workflow to address and optimize your supervised learning problems

+ wide and easy-to-get overview on most essential concepts

+ improves your understanding of NN; those who are already familiar with these concepts might still benefit from this clear and insightfull presentation

cons: - programming assignment will not suffices to give you a sufficient knowledge of tensorflow to make your own applications, you should integrate a bit. (However, mastering tensorflow is not the intention of the assignment).

By Maxime

•

Sep 8, 2020

Très bon complément de la partie 1 avec systématiquement une présentation intuitive puis technique (pour programmer à partir de 0) des concepts ..Le contenu est très dense, je conseille donc de prendre des notes et d'essayer de refaire quelque démonstration! Je conseille de bien lire tous les codes en détail ligne à ligne.

Par contre l'introduction à tensor flow n'est pas très bien expliquée selon moi (seul petit point faible) mais ce n'est pas l'objet de cette partie je pense.

Attention la deuxième partie ne comporte pas de sous titre français. Mais finalement en ayant un niveau moyen en anglais, j'ai facilement suivi avec les sous titre anglais car il y a souvent des schéma et le même vocabulaire revient souvent.

By Baohe Z

•

Oct 5, 2017

Good pace for beginner as the last one. With step by step teaching us a lot of useful skills to train our model much faster, Andrew starts to put more attention on practical field, and rather than giving us many equations, he as before likes to use some vivid examples for giving us an intuition, which I think is very helpful to understand those scientific words of computer science. But it doesn't mean, that this course is perfect, even I gave a full point to it. The subtitles have a lot of mistakes and the audio is also poorly processed. Sometimes, you will hear the same words twice.

But in a word, this is the best course for the beginners and the engineers who are willing to know something about ML and AI.

By Amilkar A H M

•

Nov 23, 2018

I loved it. It showed me practical aspects of machine learning, including how to chose the hyperparameters and how to use tensor flow. My only complain is that I'm not sure how much of this information I will retain given that the practical exercises are guided. They build a lot of the functions for you. Still I'm giving it 5 stars because I have not seen this problem solved so far in any other Coursera course. They need to find a balance between teaching you a lot and making it easy enough for most people to be able to pass the exam and not get stuck in the details. Probably they could offer extra practice automatically graded exercises for those of us who want to make sure we won't forget the material.

By Victoria D

•

Nov 25, 2019

I'd highly recommend this course to any of my colleagues interested in Deep Learning.

It is a great followup to Deep Learning and Neural Networks.

My one 'complaint' is that the mathematical depth is too shallow for someone like me (PhD, Mathematical, Computational and Experimental Physics)

It would be great if there was a course that was targeted to people with advanced STEM degrees, and years and years ( 4 decades in my case) of software engineering experience, where more time was spent on the mathematical framework, and the basic algorithms; that way, I'd have the satisfaction and pleasure of constructing the full algorithm implementations myself.

That being said, once again, Andrew is a great teacher.

By Matthew J C

•

Feb 28, 2018

I was very impressed with the quality of Dr.Ng's teaching; simple enough to build confidence in your understanding of the inner workings of neural networks yet thorough enough to prepare you for deeper study (academic or otherwise). For $50 this course is a steal; you could go gather all the information & struggle through it yourself but be prepared to spend a lot of time & effort sifting through mis-information.

After taking the 1st coarse I was impressed; course 2 follows in a similar vain. Some of the courses offered through Coursera are more polished than others; if you're at all curious in deep learning, or even if you've already begun your studies, do NOT miss out on this opportunity.

By Shazib S

•

Oct 8, 2020

Perfect teaching material and syllabus. The lectures are very easy to understand and the way Andrew takes the student through each topic creates a level of understanding I did not have before. Thank you Andrew Ng, Coursera, and Deeplearning.ai

One thing I would like to say is that there are quite a few problems with the sound, especially the trailing emphasis on the 'sss' sounds is very annoying and distracting. Also, there should have been a pointer visible on the screen. As the videos now are, when Andrew says something like "... for this equation here..." I have no idea which equation he is talking about. Kindly resolve this problem. Otherwise this course is 10/10.

By Francis S

•

Aug 26, 2019

Previously, I have taken online classes before in Machine Learning by going the cheap route (Udemy, blogs, youtube) and you get what you pay for. Andrew Ng explains it the most thorough, easiest, and simplest way possible. Presentation material is very understandable. Great class for new machine learning learners. Highly recommend it. The only downside is that the programming exercises are little too easy in my opinion. I feel like the best way to get your hands dirty is to do actual projects (do your own projects). These lectures are good for intuition and background of different types of Neural Network architectures. Other than that, Great material. Thanks Andrew!