From the course: Machine Learning Foundations: Linear Algebra

Changing basis of vectors

- Numerous supplied machine learning problems can be reduced by changing from one coordinate system to another coordinate system, which is basically the same as changing from one basis to another. Up until now we have learned that the vector is an object that takes you from the origin into some point in space. A coordinate system is defined by unit vectors that we have named I and J. Now, we are going to define them as 'e one' and 'e two'. These unit factors have coordinates one zero and zero one respectively. Every vector in space is a unique combination of these basis vectors. Let's define vector 'a' that will be four units along 'e one' and three units down along 'e two'. So, vector 'a' is equal to vector sum four 'e one', plus minus three 'e two', or we can write it down as a list, four minus three. Basis vectors, form basis for space. And any vector in this space can simply be written as a linear combination of these two vectors. There are three properties of basis vectors. They're linearly independent of each other, meaning if you take vector 'e one' and multiply it with a scalar, you cannot get a vector 'e two'. The same goes vice versa for vector 'e two'. They spend the whole space, meaning that any vector in that space can be written as a linear combination of these two vectors. They aren't unique. We can find many sets of basis vectors. I could also describe a vector 'a' as a sum of some other vectors that I could use to define the space. So, I could have another set of vectors 'f' and define them as 'f one' that will have coordinates to one, and 'f two that will have coordinates minus one, two. And we can then define vector 'a' by using vectors 'f one' and 'f two'. That is where dot product comes into play. Because we can use it to find the numbers for vector 'a' in the new basis 'f'. If we can make a vector projection of a vector 'a', on 'f one' and a vector projection on 'f two', and when we sum these two vector projections, we can get exactly vector 'a' in the set of 'f' basis vector. Then vector a will have some scalar multiplied with 'f one', plus some scalar multiplied with 'f two'. We can write it down as, a equals lambda one 'f one', plus lambda two 'f two'. In order to change the basis vectors 'f one' and 'f two', have to be orthogonal. We can check it in our example by calculating the dot product. In our case, we have two multiplied with minus one, plus one multiplied with two equals zero, it's zero. So our vectors are 90 degrees orthogonal. Great. Now, let's calculate vector projection of 'a' on 'f one', it's equal dot product of 'a' an 'f one', divided by the length of 'f one'. We have four multiplied with two, plus one multiplied with minus three, divided by two squared plus one squared. We get one. And when we multiply it with 'f one', we get two, one. Similarly, we have to calculate vector projection of 'a' on 'f two'. It's equal four multiplied with minus one, plus two multiplied with minus three, divided by minus one squared plus two squared. We get minus two. And when we multiply it with 'f two', we get two minus four. Now, we can write vector 'a', four minus three as one multiplied with two, one, plus minus two multiplied with minus one, two. So, we have covered our 'e' set of basis vectors to 'f' set of basis vectors, using a simple procedure called changing basis of vectors.

Contents