(2) Weighted Slope One

Let’s say we want to predict how well user 𝑣 will like item 𝑘. It’s a 2-step process:
Let’s say we’re trying to find the predicted rating of Deadmau5 for Angelica
Let’s say we’re trying to find the predicted rating of Deadmau5 for Angelica
 Matrix Factorization
 What is better is to have something interesting recommended to us automatically without asking.
 Indeed, from as simple as a list of the most popular bookmarks on Delicious, to some more personalized
recommendations we received on Amazon, we are usually offered recommendations on the Web.
 While user-based or item-based collaborative filtering methods are simple and intuitive, matrix factorization
techniques are usually more effective because they allow us to discover the latent features underlying the
interactions between users and items
 matrix factorization is simply a mathematical tool for playing around with matrices, and is therefore applicable
in many scenarios where one would like to find out something hidden under the data.
 as its name suggests, matrix factorization is to, obviously, factorize a matrix, i.e. to find out two (or more)
matrices such that when you multiply them you will get back the original matrix.
 matrix factorization can be used to discover latent features underlying the interactions between two
different kinds of entities.
 (Of course, you can consider more than two kinds of entities and you will be dealing with tensor factorization,
which would be more complicated.) 
 Matrix Factorization Based Recommendation Systems
• Matrix Factorization* essentially seeks to map users and items to a latent factor space,
such that user-item interactions are modeled as inner products in that space.
Matrix Factorization Based Recommendation Systems
 Matrix Factorization
• The intuition behind using matrix factorization to solve this problem is that there should be
some latent features that determine how a user rates an item.
• For example, two users would give high ratings to a certain movie if they both like the
actors/actresses of the movie, or if the movie is an action movie, which is a genre preferred
by both users.
• Hence, if we can discover these latent features, we should be able to predict a rating with
respect to a certain user and a certain item, because the features associated with the user
should match with the features associated with the item.

 In trying to discover the different features, we also make the assumption that the number
of features would be smaller than the number of users and the number of items.
 It should not be difficult to understand this assumption because clearly it would not be
reasonable to assume that each user is associated with a unique feature (although this is
not impossible).
 Matrix Factorization Based Recommendation Systems

 Assume we have the customers’ ranking table of 5 users and 5 movies, and the ratings are
integers ranging from 1 to 5, the matrix is provided by the table below.
Since not every user gives ratings to all the movies,
there are many missing values in the matrix and it
results in a sparse matrix. Hence, the null values not
given by the users would be filled with 0 such that
the filled values are provided for the multiplication

 For example, two users give high ratings to a certain movie when the movie is acted by their favorite actor and
actress or the movie genre is an action one, etc.
 From the table above, we can find that the user1 and user3 both give high ratings to move2 and movie3.
 from the matrix factorization, we are able to discover these latent features to give a prediction on a rating
with respect to the similarity in user’s preferences and interactions.
 Given a scenario, user 4 didn’t give a rating to the movie 4. We’d like to know if user 4 would like movie 4.
 The method is to discover other users with similar preferences of user 4 by taking the ratings given by
users of similar preferences to the movie 4 and predict whether the user 4 would like the movie 4 or not.
Matrix Factorization as Feature Engineering in Recommender Systems

Suppose we have a data set which contains the items ratings given by various users. This is
a typical recommender systems problem where your job is do recommend new items to the
i^th user based on the previous items i^th user has rated.

Let’s take n → number of users, m → number of items then our Rating Matrix will be of
the order of (nxm)
Mathematic concept of matrix factorization
 Define a set of Users (U), items (D), R size of |U|, and |D|.
 The matrix |U|*|D| includes all the ratings given by users.
 The goal is to discover K latent features.
 Given with the input of two matrices P (|U|*k) and Q (|D|*k), it would generate the product
result R.

 Matrix P represents the association between a user and the features while matrix Q
represents the association between an item and the features. 
 We can get the prediction of a rating of an item by the calculation of the dot product of
the two vectors corresponding to u_i and d_j.
Example of matrix factorization

• The dot product of user and item matrix can generate the rating matrix, while the user matrix
is the shape of k (users) * f (features) and the item matrix is the shape of j(items) * f (features).
• From user’s and item’s matrices, features of the movies can be its genre, actors, plot, etc.
• Having two features of the factored matrices, let’s assume F1 to be “If this movie is a comedy or
not?” and F2 to be “if Robin Williams acts in the movie?”
• User Matrix: According to User1, if its a comedy movie, he’ll give it 4 points and if Robin
Williams is the actor in the movie, he’ll give it 3 more points.
• Item Matrix: There are mainly binary values in the item matrix where the value is 1 when it meets
the conditions of features above and 0 otherwise.
• By performing the dot product of the user matrix and item matrix, the rating matrix would be
generated.
• The matrix factorization of user and item matrices can be generated when the cost
function RMSE is minimized through matrix factorization.
• Following the above mathematic concept, gradient descent is one of the methods to minimize
RMSE through each iteration.
Dot Products
The dot product of two length-n vectors a and b is defined as:
a⋅b=∑aibi=a1b1+a2b2+⋯+anbn
The result is a single scalar number (not a vector).
The dot product is only defined for vectors of the same length. This means we need to
use the same size for movie embeddings and user embeddings.
As an example, suppose we've trained embeddings of size 4, and the
movie Twister is represented by the vector:
mTwister=[1.0,−0.5,0.3,−0.1]
And the user Stanley is represented by:
uStanley=[−0.2,1.5,−0.1,0.9]
What rating do we think Stanley will give to Twister? We can calculate our model's output as:
mTwister⋅uStanley=(1.0*−0.2)+(−0.5*1.5)+(0.3*−0.1)+(−0.1*0.9) =−1.07

If overall average rating in the training set (about 3.5). So we predict that Stanley will give Twister 3.5+
(−1.07)=2.433.5+(−1.07)=2.43 stars.
Conclusion:
• Matrix factorization is a collaborative filtering method to find the
relationship between items’ and users’ entities. Latent features, the
association between users and movies matrices, are determined to find
similarity and make a prediction based on both item and user entities
• The matrix factorization of user and item matrices can be generated when
the math cost function RMSE is minimized through matrix factorization.
Gradient descent is a method to minimize the cost function.
Singular Value Decomposition (SVD)
 SVD stands for “singular value decomposition”.
 it enables to transform data (set of vectors of the same dimension and specified category)
to space where it is easy to distinguish among the categories.
 It is a matrix factorization technique where a matrix is decomposed into a product of a
square matrix, a diagonal (possible rectangular) matrix, and another square matrix.
 The diagonal matrix contains the “singular values”, the square roots of the eigenvectors
of MH⋅M where M is your original matrix and MH is the hermitian (if you’re dealing with real
numbers, it’s the same as MT).
 The square matrices contain the corresponding eigenvectors of [Link] and of MH⋅M
 It is often used to get a low-rank approximation of a matrix. You find the highest k elements
of the diagonal matrix and drop all other columns and rows, and do the same in the square
matrices
Singular Value Decomposition (SVD)
SVD factors a single matrix into matrix U, D and V* respectively.

The SVD can also be seen as the

decomposition of one complex
transformation in 3 simpler
transformations (rotation, scaling,
and rotation).

So basically it allows us to express

•U and V* are orthogonal matrices. our original matrix as a linear
•D is a diagonal matrix of singular values. combination of low-rank matrices.
Imagine you have an online store with N customers and L items for sale. You keep track of what customers have
bought what items. You keep all of that in a matrix M of dimensions N×L. Each row corresponds to a customer,
each column to an item.

 Now, how would we calculate what customers buy the same kind of things as a given customer? Well, that’s
easy: you calculate [Link] .
 Each row of this will tell you, for each other customer, how many items they both bought. If this is high, they
are very similar customers. You can do the same with the items, but here you use MH . M

many data generation processes are sloppy, meaning they strongly depend only on a few parameters or their
combination. Which means the data generated by such processes sits in a low dimensional surface that is
embedded in a space with larger number of dimension. If that surface happens to be flat then all you need is a
few basis vectors that describe the flat surface. SVD calculates these basis vectors.
 Matrix Factorization Based Recommendation Systems

• Recall that there are two key approaches to collaborative filtering:

 Neighborhood Methods compute relationships between users (user-
based) or between items (item-based)
 Latent Factor models characterize users and items based on latent factors
inferred from rating patterns (using matrix factorization)

Note: Latent Variable are variables that are not directly observed but are rather inferred through a 
mathematical model from other variables that are observed (directly measured). Mathematical models
that aim to explain observed variables in terms of latent variables are called latent variable models.