Applications of linear algebra in ML

- [Instructor] Linear algebra is one of the main building blocks of machine learning. At its base, it is a study of vectors, matrices, and linear functions. So you have to be familiar with vectors, matrices, and their characteristics, as you're going to deal with them in ML. What about machine learning algorithms? Without knowing linear algebra, you cannot understand them and therefore cannot change and improve them. Let's explore nine most important applications of linear algebra. Data set and data files. In machine learning we fit the model on a data set. This data set is either a matrix or a vector. For example, our model could be a fitness related model that predicts quality of sleep. Images and photographs. If you're working on a computer vision application, then you're dealing with images or photographs. In this case, you cannot send an image to a model and expect it to understand it. Each image is made of pixels that are basically just colored squares of varying intensities. If you have a black and white image then it's a single pixel. And for colored images, it's three pixel values for RGB. All images are stored as a matrix. Each operation that is performed on the image from cropping, scaling, et cetera, is described using the notation and operations of linear algebra. Data preparation. Data preparation, such as dimensionality reduction and one hot encoding requires linear algebra. Usually we come across data that is made up of thousands of variables and our model becomes extremely complicated. That's when dimensionality reduction comes into play. Data sets are represented as matrices and then we can use matrix factorization methods to reduce it into its constituent parts. One hot encoding is used when working with categorical data, such as the class labels for classification problems or categorical input variables. It is common to encode categorical variables to make them easier to work with. Linear regression. It is usually used in machine learning for predicting numerical values in simple regression problems. The most common way of solving linear regression is via the least squares optimization that is solved using matrix factorization methods from linear regression. Regularization. Overfitting is one of the greatest obstacles in machine learning. When a model is too close a fit for the available data to the point that it does not perform well with any new or outside data. Regularization is a concept from linear algebra that is used to prevent the model from overfitting. Simple models are models that have smaller coefficient values. Regularization is a technique that is often used to encourage a model to minimize the size of coefficients while it's being fit on data. Principle component analysis. Modeling data with many features is challenging and it's hard to know which features of data are relevant and which are not. One of the methods for automatic reducing the number of columns of a data set is principle component analysis. This method is used in machine learning to create projections of high dimensional data for both visualization and for training models. The core of the PCA method is a metric factorization method. Latent semantic analysis. LSA is a form of data preparation used in natural language processing, a subfield of machine learning for working with text data. In this case, documents are usually represented as a large matrices of word occurrences. Then we can apply matrix factorization methods to them in order to be able to easily compare, query, and use them as the basis for the ML model. Recommender systems. Recommender systems are predictive modeling problems that involve the recommendation of products. They are used each time you buy something on Amazon or a similar shop and you get recommendations of products based on your previous purchases. Deep learning. Deep learning is a specific subfield of machine learning. Scaled up to multiple dimensions, deep learning methods work with vectors, matrices, and tensors of inputs and coefficients.