>>> By enrolling in this course you agree to the End User License Agreement as set out in the FAQ. Once enrolled you can access the license in the Resources area <<< This course, Applied Artificial Intelligence with DeepLearning, is part of the IBM Advanced Data Science Certificate which IBM is currently creating and gives you easy access to the invaluable insights into Deep Learning models used by experts in Natural Language Processing, Computer Vision, Time Series Analysis, and many other disciplines. We’ll learn about the fundamentals of Linear Algebra and Neural Networks. Then we introduce the most popular DeepLearning Frameworks like Keras, TensorFlow, PyTorch, DeepLearning4J and Apache SystemML. Keras and TensorFlow are making up the greatest portion of this course. We learn about Anomaly Detection, Time Series Forecasting, Image Recognition and Natural Language Processing by building up models using Keras one real-life examples from IoT (Internet of Things), Financial Marked Data, Literature or Image Databases. Finally, we learn how to scale those artificial brains using Kubernetes, Apache Spark and GPUs. IMPORTANT: THIS COURSE ALONE IS NOT SUFFICIENT TO OBTAIN THE "IBM Watson IoT Certified Data Scientist certificate". You need to take three other courses where two of them are currently built. The Specialization will be ready late spring, early summer 2018 Using these approaches, no matter what your skill levels in topics you would like to master, you can change your thinking and change your life. If you’re already an expert, this peep under the mental hood will give your ideas for turbocharging successful creation and deployment of DeepLearning models. If you’re struggling, you’ll see a structured treasure trove of practical techniques that walk you through what you need to do to get on track. If you’ve ever wanted to become better at anything, this course will help serve as your guide. Prerequisites: Some coding skills are necessary. Preferably python, but any other programming language will do fine. Also some basic understanding of math (linear algebra) is a plus, but we will cover that part in the first week as well. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge. To find out more about IBM digital badges follow the link ibm.biz/badging.
Recurrent neural networks
Loading...
Из курса от партнера IBM
Applied AI with DeepLearning
147 оценки
Из урока
deep learning frameworks
Познакомьтесь с преподавателями
Romeo KienzlerChief Data Scientist, Course Lead
IBM Watson IoT
Niketan PansareSenior Software Engineer
IBM Research
Tom HanlonTraining Director
Skymind
Max PumperlaDeep Learning Engineer
Ilja RasinData Scientist
IBM Watson Health
All right. In this video,
we're going to introduce the Recurrent Neural Networks in Keras,
and in particular talk about an esteems.
As an application, we're going to classify sentiments from movie reviews.
So there's a few architectures available for Recurrent Neural Networks in Keras.
The first one is a class called simpler RNN,
which is a basic plane on the nose recurrent neural network,
which suffers from problems like vanishing and exploding radiant.
So you see those very rarely used in practice
that gated recursive units introduced in 2014,
has certainly its used cases.
Also, an LSTM are
a long short term memory models introduced in 1997 by Huffington Schmidhuber.
Those are the most popular Recurrent Neural Networks in Keras in maybe overall.
So, oftentimes when people talk about using RNNs usually they mean finding LSTMs.
We'll focus in this lecture exclusively on LSTMs.
Alright. How do we build LSTM layers with Keras?
There's a few things we have to specify.
Recall that with LSTM layer,
you have two sets of weights.
First, regular set of weights and then also recurrent set of weights.
So in Keras that's called recurrent RNN.
And apart from specifying the units,
so the number of units and then activation fronting for all layer,
we also have to specify everything recurrent.
So for instance we need a recurrent initializer or recurrent activation and so on.
So if you want to specialize or specify a regularizer or constraint, you can do that.
Something of note here is that LSTM layers
allows you to specify a dropout within the layer.
So you don't have to specify a dropout layer separately,
but you can specify a dropout rate for a LSTM.
Both for direct set of weights,
specified by dropout and for the recurrent weights specified by recurrent neural network.
The last key words are we mention here is
return sequences which are set True, False, by default.
So, what that means is if you started to true,
then return of your LSTM won't be a simple vector but a matrix instead.
So, where you run an LSTM is by you apply different time sets.
Right? You go through different time sets and at the end the output is one,
one back to usual.
If you set return sequences to true,
you will also get returns,
all at the immediate values adding up at the matrix time.
Okay. To compute our application we need to introduce
another layer which is quite useful for many such applications.
And so embedding letters,
are used as a very first layer in Keras.
And what you use them for is you transform integers into vectors and like.
To you give you one example,
these two numbers are three and 12,
those can be embedded into vectors of length two,
which are on the right.
A prototypical example that you would let us join which is going to do,
is we want to embed a certain vocabulary into a vector space,
meaning each word in those vocabulary has to be
embedded or has to be given a representation of a vector space,
and then you apply those embedments to sequences of words.
This is when you can map a 2D input to a 3D output which connects to LSTM.
So meaning, if you have a mini-batch of sequences of IDs,
you can map those to mini-batch of sequences of vectors by a mini-batch of matrices.
And this can be set into an LSTM.
Alright. How do we initialize on embedding with Keras?
There's a few things we have to specify.
First off, the input dimension is
our vocabulary size that we have to define before length.
We also need to specify the output vector length, of the day.
In our example from the previous lines,
which show two dimensional vectors that usually returns
much larger vectors as I put in dimension.
Third item we need to specify here,
is this so called embeddings initializes.
So the weights of embedding in
Keras are called Embeddings Voids and we initialize them like that.
A false key word what's quite interesting is the mask zero flag.
So, your input sequences in an embedding layer,
may or may not have different lengths.
So, if you think about sentences of different length,
and we can use the value zero as a special value that we can then mask out.
So for instance, you start out with sequences of various length,
and then you paart those sequences with zeros to make them of the same size.
Only do them mask out the zero values.
Okay.
Our use case here is sentiment classification from movie reviews,
which is a database of 25000 movie reviews from IMDB.
Those are labeled either as good or bad.
The data status is also visible through the Keras dataset module.
And the good thing is that the data is already preprocessed sequences of integers.
So, we don't actually have to work with
the words vocabulary but those have already been marked as integers.
Our task then is to classify our sentiment so good or bad from the review content.
And the way we are going to tackle this is we first embed our sentences with
an embedding layer and then learn the sequential structure with an LSTM.
So how do we do this?
So we have a few imports of sequential model maxlen layer,
embedding layer and LSTM and we also import our data set.
So first off, we spent specifying the maximum number of features that goes to 20,000.
So meaning, we only choose to model 20000 most common items from our vocabulary.
We specify a maximum length for our features.
So, we only allow sequences of length 80 which would be quite short.
With movie reviews, that's how we choose.
So then we can simply load IMDB data into memory,
by imposing IMDB low data with
the maximum of features as specified to get back training and test data.
Alright. Next, we pad on sequences according to the maxlength we have specified.
So that would mean if the sequence training and
testing of data sets is shorter than 80 items,
we would pad them with zeros at the end,
and if they're longer, we crack them.
Next, we initialize the sequential model and add on
the embedding layer with the given maximum number of features and 128 equal dimension.
This then connects to LSTM layer with also 128 equal dimension,
and we also specify 20 percent dropout rate for both regular and recurring weights.
At very end, we are at the dense layer of the single output dimension and
the sigmoid function which would specify a value between zero and one,
zero meaning bad and one good.
What's left is to compile them all which we again do with binary crossentropy,
suppress the gradient and we also have the same accuracy as before.
Then we set our model with batch size 32 and let it run for 15 epochs,
and again we use our test service validation data.
Afterwards, we can evaluate our model.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
