0:00

The ability of python to allow you to use broadcasting operations and

Â more generally, the great flexibility of the python numpy program language is,

Â I think, both a strength as well as a weakness of the programming language.

Â I think it's a strength because they create expressivity of the language.

Â A great flexibility of the language lets you get a lot done even with just a single

Â line of code.

Â But there's also weakness because with broadcasting and this great amount of

Â flexibility, sometimes it's possible you can introduce very subtle bugs or

Â very strange looking bugs, if you're not familiar with all of the intricacies of

Â how broadcasting and how features like broadcasting work.

Â For example, if you take a column vector and add it to a row vector, you would

Â expect it to throw up a dimension mismatch or type error or something.

Â But you might actually get back a matrix as a sum of a row vector and

Â a column vector.

Â So there is an internal logic to these strange effects of Python.

Â But if you're not familiar with Python, I've seen some students have very strange,

Â very hard to find bugs.

Â So what I want to do in this video is share with you some couple tips and

Â tricks that have been very useful for me to eliminate or

Â simplify and eliminate all the strange looking bugs in my own code.

Â And I hope that with these tips and tricks,

Â you'll also be able to much more easily write bug-free, python and numpy code.

Â 1:25

To illustrate one of the less intuitive effects of Python-Numpy,

Â especially how you construct vectors in Python-Numpy, let me do a quick demo.

Â Let's set a = np.random.randn(5),

Â so this creates five random Gaussian

Â variables stored in array a.

Â And so let's print(a) and now it turns out that

Â the shape of a when you do this is this five color structure.

Â And so this is called a rank 1 array in Python and

Â it's neither a row vector nor a column vector.

Â And this leads it to have some slightly non-intuitive effects.

Â So for example, if I print a transpose, it ends up looking the same as a.

Â So a and a transpose end up looking the same.

Â And if I print the inner product between a and a transpose, you might think

Â a times a transpose is maybe the outer product should give you matrix maybe.

Â But if I do that, you instead get back a number.

Â So what I would recommend is that when you're coding new networks,

Â that you just not use data structures where the shape is 5, or n, rank 1 array.

Â Instead, if you set a to be this, (5,1),

Â then this commits a to be (5,1) column vector.

Â And whereas previously, a and a transpose looked the same,

Â it becomes now a transpose, now a transpose is a row vector.

Â Notice one subtle difference.

Â In this data structure, there are two square brackets when we print a transpose.

Â Whereas previously, there was one square bracket.

Â So that's the difference between this is really a 1 by

Â 5 matrix versus one of these rank 1 arrays.

Â And if you print, say, the product between a and a transpose,

Â then this gives you the outer product of a vector, right?

Â And so, the outer product of a vector gives you a matrix.

Â So, let's look in greater detail at what we just saw here.

Â The first command that we ran, just now, was this.

Â And this created a data structure with

Â a.shape was this funny thing (5,) so

Â this is called a rank 1 array.

Â And this is a very funny data structure.

Â It doesn't behave consistently as either a row vector nor a column vector,

Â which makes some of its effects nonintuitive.

Â So what I'm going to recommend is that when you're doing your programing

Â exercises, or in fact when you're implementing logistic regression or

Â neural networks that you just do not use these rank 1 arrays.

Â 4:21

Instead, if every time you create an array,

Â you commit to making it either a column vector, so

Â this creates a (5,1) vector, or commit to making it a row vector,

Â then the behavior of your vectors may be easier to understand.

Â So in this case, a.shape is going to be equal to 5,1.

Â And so this behaves a lot like a, but in fact, this is a column vector.

Â And that's why you can think of this as (5,1) matrix, where it's a column vector.

Â And here a.shape is going to be 1,5,

Â and this behaves consistently as a row vector.

Â 5:02

So when you need a vector, I would say either use this or this, but

Â not a rank 1 array.

Â One more thing that I do a lot in my code is if I'm not entirely sure what's

Â the dimension of one of my vectors, I'll often throw in an assertion statement

Â like this, to make sure, in this case, that this is a (5,1) vector.

Â So this is a column vector.

Â These assertions are really Set to execute, and

Â they also help to serve as documentation for your code.

Â So don't hesitate to throw in assertion statements like this whenever you

Â feel like.

Â And then finally, if for some reason you do end up with a rank 1 array,

Â You can reshape this, a equals a.reshape

Â into say a (5,1) array or a (1,5) array so

Â that it behaves more consistently as either column vector or row vector.

Â So I've sometimes seen students end up with very hard to track

Â because those are the nonintuitive effects of rank 1 arrays.

Â By eliminating rank 1 arrays in my old code, I think my code became simpler.

Â And I did not actually find it restrictive in terms of things I could

Â express in code.

Â I just never used a rank 1 array.

Â And so takeaways are to simplify your code, don't use rank 1 arrays.

Â Always use either n by one matrices,

Â basically column vectors, or one by n matrices, or basically row vectors.

Â Feel free to toss a lot of insertion statements, so

Â double-check the dimensions of your matrices and arrays.

Â And also, don't be shy about calling the reshape operation to make sure that your

Â matrices or your vectors are the dimension that you need it to be.

Â So that,

Â I hope that this set of suggestions helps you to eliminate a cause of bugs

Â from Python code, and makes the problem exercise easier for you to complete.

Â