In the last couple of weeks,

you've looked at creating neural networks

to forecast time-series data.

You started with some simple analytical techniques,

which you then extend it to using

Machine Learning to do a simple regression.

From there you use the DNN that you

tweaked a bit to get an even better model.

This week, we're going to look at

RNNs for the task of prediction.

A Recurrent Neural Network,

or RNN is a neural network

that contains recurrent layers.

These are designed to

sequentially processes sequence of inputs.

RNNs are pretty flexible,

able to process all kinds of sequences.

As you saw in the previous course,

they could've been used for predicting text.

Here we'll use them to process the time series.

This example, will build an RNN that contains

two recurrent layers and a final dense layer,

which will serve as the output.

With an RNN, you can feed it in batches of sequences,

and it will output a batch of forecasts,

just like we did last week.

One difference will be that

the full input shape when

using RNNs is three-dimensional.

The first dimension will be the batch size,

the second will be the timestamps,

and the third is the dimensionality

of the inputs at each time step.

For example, if it's a univariate time series,

this value will be one,

for multivariate it'll be more.

The models you've been using to

date had two-dimensional inputs,

the batch dimension was the first,

and the second had all the input features.

But before going further,

let's dig into the RNN layers to see how they work.

What it looks like there's lots of cells,

there's actually only one,

and it's used repeatedly to compute the outputs.

In this diagram, it looks like there's lots of them,

but I'm just using the same one being

reused multiple times by the layer.

At each time step,

the memory cell takes the input value for that step.

So for example, it is zero at time zero,

and zero state input.

It then calculates the output for that step,

in this case Y0, and a state

vector H0 that's fed into the next step.

H0 is fed into the cell with X1 to produce Y1 and H1,

which is then fed into the cell at the next step

with X2 to produce Y2 and H2.

These steps will continue until

we reach the end of our input dimension,

which in this case has 30 values.

Now, this is what gives this type of architecture

the name a recurrent neural network,

because the values recur due to the output of the cell,

a one-step being fed back

into itself at the next time step.

As we saw in the NLP course,

this is really helpful in determining states.

The location of a word in

a sentence can determine it semantics.

Similarly, for numeric series,

things such as closer numbers in the series might have

a greater impact than those

further away from our target value.