RecommenderSystem.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recommender system\n",
    "In this notebook, we will leverage the trained Weighted Matrix Factorization model to make recommendations. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from src.wmfact import WeightedMatrixFactorization\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "# Load the model\n",
    "model = WeightedMatrixFactorization.load('./models/wmf_wals_nlat270_niter12_lambdareg0.02.pkl')\n",
    "\n",
    "# Load the data\n",
    "feedbacks = np.load('./data/feedbacks.npy') # the feedbacks matrix that was used to train the model\n",
    "movies = pd.read_csv('data/raw/movies.csv') # contains info about each movie "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cosine similarity\n",
    "Cosine similarity is a metric used to measure the similarity between two vectors in a multi-dimensional space. It calculates the cosine of the angle between the vectors, indicating the direction of similarity regardless of their magnitude.\n",
    "\n",
    "The formula for cosine similarity between vectors $A$ and $B$ is:\n",
    "\n",
    "$$ cosine\\_similarity(A, B) = \\frac{A \\cdot B}{||A|| \\times ||B||} $$\n",
    "\n",
    "- $A \\cdot B$ is the dot-product of the two vectors;\n",
    "- $||A||$ and $||B||$ are the Euclidian norms (magnitudes) of the two vectors.\n",
    "\n",
    "The cosine similarity value lies in the range $[−1,1]$, where a value closer to $1$ indicates higher similarity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cosine_similarity(vector1: np.ndarray, vector2: np.ndarray) -> float:\n",
    "    \"\"\"\n",
    "    Calculate the cosine similarity between two vectors.\n",
    "    Formula: similarity = (vector1 . vector2) / (||vector1|| * ||vector2||)\n",
    "    \n",
    "    Parameters:\n",
    "    - vector1 (numpy.ndarray): The first vector.\n",
    "    - vector2 (numpy.ndarray): The second vector.\n",
    "    \n",
    "    Returns:\n",
    "    - similarity (float): The cosine similarity between the two vectors.\n",
    "\n",
    "    Raises:\n",
    "    - ValueError: If one of the input vectors has a norm of 0.\n",
    "\n",
    "    Usage:\n",
    "    >>> similarity = cosine_similarity(vector1, vector2)\n",
    "    \"\"\"\n",
    "    norm_vector1 = np.linalg.norm(vector1)\n",
    "    norm_vector2 = np.linalg.norm(vector2)\n",
    "    \n",
    "    if norm_vector1 == 0 or norm_vector2 == 0:\n",
    "        raise ValueError(\"One of the vectors has a norm of 0.\")\n",
    "    \n",
    "    similarity = np.dot(vector1, vector2) / (norm_vector1 * norm_vector2)\n",
    "    return similarity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Content-based filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def contentbased_filtering(user_id: int, model: WeightedMatrixFactorization, top_n: int=10) -> pd.DataFrame:\n",
    "    \"\"\"\n",
    "    Get the top-n items for a user using content-based filtering.\n",
    "\n",
    "    Parameters:\n",
    "    - user_id (int): The user ID.\n",
    "    - model (WeightedMatrixFactorization): The trained model.\n",
    "    - top_n (int): The number of items to recommend. Default is 10.\n",
    "\n",
    "    Returns:\n",
    "    - top_k_movies (pandas.DataFrame): The top-k items recommended for the user.\n",
    "\n",
    "    Raises:\n",
    "    - ValueError: If the provided user ID is invalid or if it's not present in the model.\n",
    "    - ValueError: If the top-n parameter is less than or equal to 0.\n",
    "\n",
    "    Usage:\n",
    "    >>> top_k_movies = contentbased_filtering(user_id, model, top_n=10)\n",
    "    \"\"\"\n",
    "    \n",
    "    if not isinstance(user_id, int):\n",
    "        raise ValueError(\"User ID must be an integer.\")\n",
    "    \n",
    "    if user_id < 0 or user_id >= model.n_users:\n",
    "        raise ValueError(\"Invalid user ID. User ID must be within the range of existing users.\")\n",
    "\n",
    "    if not isinstance(top_n, int) or top_n <= 0:\n",
    "        raise ValueError(\"Top-n parameter must be a positive integer.\")\n",
    "\n",
    "    # get the user and item embeddings:\n",
    "    users_embedding, items_embedding = model.get_embeddings()\n",
    "\n",
    "    # get the user embedding for the user_id:\n",
    "    user_embedding = users_embedding[user_id]\n",
    "\n",
    "    # get all the items that the user has not rated (the indices of the items):\n",
    "    idx_not_rated = np.where( \n",
    "        np.isnan( feedbacks[user_id] )\n",
    "    )[0]\n",
    "\n",
    "    # compute the similarities between this user and the items that he has not rated:\n",
    "    similarities = np.array([ \n",
    "        cosine_similarity(\n",
    "            vector1 = user_embedding,           # the user embedding\n",
    "            vector2 = items_embedding[idx]      # the item embedding of the idx-th item\n",
    "        ) for idx in idx_not_rated              # iterate over non-rated items\n",
    "    ])\n",
    "\n",
    "    similarities = similarities * 100               # convert to %\n",
    "    similarities = np.round( similarities, 2 )      # round 2 decimal places\n",
    "    top_k = np.argsort(similarities)[::-1][:top_n]  # get the indices of the top-n items\n",
    "    top_k_movies = movies.iloc[top_k]               # get the corresponding movies\n",
    "    top_k_movies = top_k_movies.copy()\n",
    "    top_k_movies['similarity (%)'] = similarities[top_k] # add the similarities to dataframe\n",
    "    return top_k_movies     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now recommend top 10 movies for the user with id = 475 using **Content Based** filtering:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      movie_id                                   title  similarity (%)\n",
      "923        924                     White Squall (1996)           40.32\n",
      "135        136     Mr. Smith Goes to Washington (1939)           39.60\n",
      "353        354              Wedding Singer, The (1998)           36.73\n",
      "729        730  Queen Margot (Reine Margot, La) (1994)           36.57\n",
      "144        145               Lawnmower Man, The (1992)           36.39\n",
      "154        155                    Dirty Dancing (1987)           36.17\n",
      "1100      1101        Six Degrees of Separation (1993)           36.04\n",
      "523        524              Great Dictator, The (1940)           35.74\n",
      "487        488                     Sunset Blvd. (1950)           35.64\n",
      "669        670                   Body Snatchers (1993)           35.26\n"
     ]
    }
   ],
   "source": [
    "top_k_movies = contentbased_filtering( user_id = 475, model = model, top_n = 10 )\n",
    "print(top_k_movies)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collaborative filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def collaborative_filtering(user_id: int, model: WeightedMatrixFactorization, top_n: int=10) -> pd.DataFrame:\n",
    "    \"\"\"\n",
    "    Perform collaborative filtering to recommend items to a user.\n",
    "\n",
    "    Parameters:\n",
    "    - user_id (int): The ID of the user for whom recommendations are to be generated.\n",
    "    - model (WeightedMatrixFactorization): The trained recommendation model.\n",
    "    - top_n (int): The number of top recommendations to return. Default is 10.\n",
    "\n",
    "    Returns:\n",
    "    - recommended_items (pd.DataFrame): A DataFrame containing the top recommended items and their similarities.\n",
    "\n",
    "    Raises:\n",
    "    - ValueError: If the provided user ID is invalid or if it's not present in the model.\n",
    "    - ValueError: If the top-n parameter is less than or equal to 0.\n",
    "\n",
    "    Usage:\n",
    "    >>> recommended_items = collaborative_filtering(user_id, model, top_n=10)\n",
    "    \"\"\"\n",
    "\n",
    "    if not isinstance(user_id, int):\n",
    "        raise ValueError(\"The user ID must be an integer.\")\n",
    "    if user_id < 0 or user_id >= feedbacks.shape[0]:\n",
    "        raise ValueError(f\"The user ID {user_id} is invalid.\")\n",
    "    if top_n <= 0:\n",
    "        raise ValueError(\"The top-n parameter must be greater than 0.\")\n",
    "    \n",
    "\n",
    "    users_embedding, items_embedding = model.get_embeddings()\n",
    "    user_embedding = users_embedding[user_id]\n",
    "\n",
    "    idx_not_rated = np.where( np.isnan(\n",
    "    feedbacks[user_id]) \n",
    "    )[0]\n",
    "\n",
    "    # similarity between this user and all the other users:\n",
    "    similarities = np.array([ \n",
    "    cosine_similarity(\n",
    "        vector1 = users_embedding[idx],\n",
    "        vector2 = user_embedding\n",
    "    ) for idx in range(users_embedding.shape[0])\n",
    "    ])\n",
    "\n",
    "    sorted_indices = np.argsort(similarities)[::-1][:int(0.10*len(similarities))]\n",
    "    sorted_indices = sorted_indices[1:]  # exclude the target user\n",
    "\n",
    "    # Aggregate preferences of similar users\n",
    "    aggregated_preferences = np.sum( users_embedding[sorted_indices], axis=0 )\n",
    "\n",
    "    # Calculate cosine similarity between the target user group and items based on content features\n",
    "    similarities = np.array([ \n",
    "        cosine_similarity(\n",
    "            vector1 = aggregated_preferences,\n",
    "            vector2 = items_embedding[idx]\n",
    "        ) for idx in idx_not_rated  # iterate over non-rated items\n",
    "    ])\n",
    "\n",
    "    # sort items and similarities together according to content-based similarity in descending order\n",
    "    sorted_indices = np.argsort(similarities)[::-1][:top_n]\n",
    "    recommended_item_indices = idx_not_rated[sorted_indices]\n",
    "    recommended_items_similarities = similarities[sorted_indices]\n",
    "\n",
    "    # a dataframe with the recommended items and their similarities:\n",
    "    recommended_items = movies.iloc[recommended_item_indices]\n",
    "    recommended_items = recommended_items.copy()\n",
    "    recommended_items['similarity'] = recommended_items_similarities\n",
    "\n",
    "    return recommended_items"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now recommend top 10 movies for the user with id = 475 using **Collaborative** filtering:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                      title  similarity\n",
      "173          Raiders of the Lost Ark (1981)    0.620386\n",
      "63         Shawshank Redemption, The (1994)    0.590658\n",
      "78                     Fugitive, The (1993)    0.582695\n",
      "171         Empire Strikes Back, The (1980)    0.580510\n",
      "49                         Star Wars (1977)    0.577964\n",
      "175                           Aliens (1986)    0.570486\n",
      "356  One Flew Over the Cuckoo's Nest (1975)    0.570391\n",
      "11               Usual Suspects, The (1995)    0.569235\n",
      "97         Silence of the Lambs, The (1991)    0.565038\n",
      "143                         Die Hard (1988)    0.553481\n",
      "194                  Terminator, The (1984)    0.551391\n",
      "650                            Glory (1989)    0.548086\n",
      "180               Return of the Jedi (1983)    0.547446\n",
      "317                 Schindler's List (1993)    0.545705\n",
      "21                        Braveheart (1995)    0.545465\n",
      "190                          Amadeus (1984)    0.542731\n",
      "95        Terminator 2: Judgment Day (1991)    0.540869\n",
      "30                      Crimson Tide (1995)    0.538863\n",
      "27                         Apollo 13 (1995)    0.537938\n",
      "195               Dead Poets Society (1989)    0.529291\n"
     ]
    }
   ],
   "source": [
    "top_k_movies = collaborative_filtering(475, model, 20)\n",
    "print(top_k_movies[['title', 'similarity']])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}