From the course: Machine Learning Foundations: Linear Algebra

Calculating eigenvalues and eigenvectors

From the course: Machine Learning Foundations: Linear Algebra

Start my 1-month free trial

Calculating eigenvalues and eigenvectors

“

- [Instructor] In the complex world of linear algebra, sometimes we discover simple, straight-away techniques that help us solve equations. One of those equations is A multiplied by v equals lambda multiplied by v. This says if we multiply matrix A with some vector v, it is the same as multiplying vector v by some scalar lambda. In case this equation is true, we call we vector v eigenvector and scalar lambda associated eigenvalue of matrix A. We can provide this equation as A multiplied by v minus lambda multiplied by v equals zero. Meaning, if we subtract lambda v from Av, we get a zero vector. Finally, we can write this equation as A minus lambda I multiplied by v equals zero, where I is an identity matrix. At the end, determinant of A minus lambda I will be equal to zero. Let's see how to calculate eigenvalues and eigenvectors in the following example. We have a two-by-two matrix A that has elements 3, 4, minus 1, and 7. First step we have to make is to shift this matrix by lambda multiplied with the identity matrix. We can write this down as A minus lambda I equals zero, and calculate the values. Next step is to set the determinant of A minus lambda I equal to zero. When we simplify our equation, we get 3 minus lambda times 7 minus lambda plus 4 equals 0. After multiplying the brackets and simplifying, we get the quadratic equation, lambda minus 5 squared equals 0. That gives us solutions and roots of our equations, lambda1 equals lambda2 equals 5. These are our eigenvalues. To get the eigenvectors, we have to go back to our system A minus lambda I equals 0. That is, in our case, equal to A minus 5I equals 0, and after simplifying, we have a matrix with elements minus 2, 4, minus 1, and 2 equals 0. It is equivalent to the systems of equations. From this system, we get the solution K1 equals 2K2, and if we choose K2 equals 2, then we get a single eigenvector K1 with elements 4 and 2. Since we could choose different integers, and plug them instead of K2, in that case, we would get different eigenvectors and solutions. Understanding and knowing how eigenvector and eigenvalues are calculated is important in the future extraction techniques. One of them is principle component analysis, or PSA, which is an algorithm used to reduce dimensionality while training a machine learning model.

Contents