12_PCA.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "12 PCA.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOJZhjGwrvzQjzglp1DekN1",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sandipanpaul21/Dimensionality-Reduction-in-Python/blob/master/12_PCA.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92lpLyYe-9JA"
      },
      "source": [
        "#### **Principal Component Analysis (PCA)**\n",
        "- It is where the *data from high dimensional space is reduced to lower dimensions*\n",
        "- In simple words, we transform our features **into a lower number of artificial features** without losing much of the information. In this method, features are transformed into a set of ‘artificial features’. \n",
        "- These **‘artificial features’ are known as Principal Components**, where the first component contains most of the information that can be contained in a single ‘artificial feature’ and we are left to select the number of components in order to reduce the features. \n",
        "\n",
        "**NOTE:** Here the features are not explicitly dropped rather the variation is extracted saving the loss of data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Olp-gyIT_32p"
      },
      "source": [
        "#### **Why use PCA in the first place?**\n",
        "\n",
        "- Suppose we have a dataset having two variables and 10 number of data points. If we were asked to visualize the data points, we can do it very easily. The result is very interpretable as well.\n",
        "- Now if we try to increase the number of variables it gets **almost impossible for us to imagine a dimension higher than three-dimensions** \n",
        "- This problem we face when analyzing higher-dimensional datasets is what commonly referred to as **“The curse of dimensionality”.**\n",
        "- So we use PCA or **Principal Component Analysis**\n",
        "  1. It reduces high dimensional data to lower dimensions while capturing maximum variability of the dataset.\n",
        "  2. Data visualization is the most common application of PCA. \n",
        "  3. PCA is also used to make the training of an algorithm faster by reducing the number of dimensions of the data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oxJMKtUo_4Vm"
      },
      "source": [
        "#### **Steps to Compute Principal Components from Scratch**\n",
        "Import Dataframe\n",
        "  1. Standardize each column\n",
        "  2. Compute Covariance Matrix\n",
        "  3. Compute Eigen values and Eigen Vectors\n",
        "  4. Derive Principal Component Features by taking dot product of eigen vector and standardized columns\n",
        "  5. PCA Dataframe\n",
        "\n",
        "**Eigen Value**\n",
        "- ‘Eigen’ is a German word that means ‘proper’ or ‘characteristic’. \n",
        "- Therefore, the term eigenvalue can be termed as characteristic value, characteristic root, proper values or latent roots as well. \n",
        "- In simple words, the eigenvalue is a scalar that is used to transform the eigenvector\n",
        "\n",
        "                                  Basic Equation is Ax = λx\n",
        "\n",
        "- The number or **scalar value “λ” is an eigenvalue of A.**\n",
        "- In Mathematics, an eigen vector corresponds to the real **non zero eigenvalues which point in the direction stretched by the transformation** whereas eigen value is considered as a factor by which it is stretched.\n",
        "- In case, **if the eigen value is negative, the direction of the transformation is negative.**\n",
        "- Eigenvalues are coefficients applied to eigenvectors that give the vectors their length or magnitude. For example, a negative eigenvalue may reverse the direction of the eigenvector as part of scaling it.\n",
        "\n",
        "**Eigen Vectors**\n",
        "- Eigenvectors are the vectors(non-zero) that do not change the direction when any linear transformation is applied. \n",
        "- **Eigen Vector changes by only a scalar factor (also known as Eigen Value)** \n",
        "\n",
        "\n",
        "Let us say A is an “n × n” matrix and λ is an eigenvalue of matrix A, then x, a non-zero vector, is called as eigenvector if it satisfies the given below expression;\n",
        "\n",
        "                           Basic Equation is Ax = λx where x is an eigenvector of A corresponding to eigenvalue, λ.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X6_em0uFXlWp"
      },
      "source": [
        "#### **Why use Eigenvalues & Eigenvectors**\n",
        "- In simple words, the concept of Eigenvectors and Eigenvalues are used to determine a set of important variables (in form of vector) along with scale along different dimensions (key dimensions based on variance) for analysing the data in a better manner.\n",
        "\n",
        "**Example 1 : Image of Human Being**\n",
        "  1. When you look at the above picture (data) and identify it as human being\n",
        "  2. What are some of the key information (dimensions / principal components) you use to call it out as human being? \n",
        "  3. Is it not body, face, legs etc information? These principal components / dimensions can be seen as eigenvector with each one of them having their own elements. \n",
        "  4. For example, body will have elements such as color, built, shape etc. \n",
        "  5. Face will have elements such as nose, eyes, color etc. \n",
        "  6. The overall data (image) can be seen as transformation matrix. \n",
        "  7. The data (transformatio matrix) when acted on the eigenvectors (principal components) will result in the eigenvectors multiplied by scale factor (eigenvalue). \n",
        "  8. And accordingly, identify the image as the tiger.\n",
        "\n",
        "**Example 2: Predicting the stock prices** \n",
        "  1. Here the dependent value is stock price and there are a large number of independent variables on which the stock price depends. \n",
        "  2. Using large number of independent variables (also called features), training one or more machine learning models for predicting the stock price will be computationally intensive. \n",
        "  3. Such models turn out to be complex models. \n",
        "\n",
        "Can we use the information stored in these variables and **extract a smaller set of variables (features) to train the models** and do the prediction while ensuring that most of the information contained in the original variables is retained / maintained. \n",
        "\n",
        "This will result in simpler and computationally efficient models. **This is where eigenvalues and eigenvectors comes into picture.**\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yl4ay6ul5XV1"
      },
      "source": [
        "# Libraries\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "from sklearn import datasets\n",
        "from sklearn.decomposition import PCA\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S3XMClV_ItPi",
        "outputId": "c9c1037d-aa20-4e29-a84d-97b47b5be1d9"
      },
      "source": [
        "# Let's take an Example\n",
        "X = np.array([1,2,3,4,5])\n",
        "Y = np.array([123,142,163,174,195])\n",
        "Z = np.array([531,522,573,514,595])\n",
        "data = pd.DataFrame({'X':X, 'Y':Y, 'Z':Z})\n",
        "print(\"Shape of Data:\", data.shape)\n",
        "print(data)\n",
        "\n",
        "# Manual Steps\n",
        "print(\"\\nStep 1. Calculate Mean of Each Column\")\n",
        "Mean_X = data[\"X\"].mean()\n",
        "print(\"Mean of X :\",Mean_X)\n",
        "Mean_Y = data[\"Y\"].mean()\n",
        "print(\"Mean of Y :\",Mean_Y)\n",
        "Mean_Z = data[\"Z\"].mean()\n",
        "print(\"Mean of Z :\",Mean_Z)\n",
        "\n",
        "print(\"\\nStep 2. Calculate Variance\")\n",
        "# Variance = sum(((x(i) - mean(x))^2))/n-1\n",
        "# Var(X) = [ (1–3.0)^2 + (2–3.0)^2 + (3-3.0)^2 + (4-3.0)^2 +(5-3.0)^2 ] / (5-1) = (4.0 + 1.0 + 0.0 + 1.0 + 4.0) / 4 = 10.0 / 4 = 2.5\n",
        "print(data.var())\n",
        "\n",
        "print(\"\\nStep 3. Covariance Matrix :\",data.cov().shape)\n",
        "# Covariance (XY) = (sum|X(i)-mean(X) * Y(i)-mean(Y)|) / (n-1)\n",
        "# Covar(XY) = [(1–3.0)*(123-159.4) + (2–3.0)*(142-159.4) + (3-3.0)*(163-159.4) + (4-3.0)*(174-159.4) + (5-3.0)*(195-159.4)] / (5-1) \n",
        "# = [2.0*36.4 + 1.0*17.4 + 0.0 + 1.0*14.5 + 2.0*35.5] / 4 \n",
        "# = 175.7 / 4 = 43.92 ~ 44\n",
        "# Covar(XY) = 44\n",
        "# If you examine the calculations carefully, you’ll see the pattern to compute the covariance of the XZ and YZ columns. \n",
        "# And you’ll see that Covar(XY) = Covar(YX).\n",
        "print(data.cov())\n",
        "\n",
        "print(\"\\nStep 4. Eigenvalues and Eigenvectors for the calculated Covariance matrix.\")\n",
        "#Calculating Eigenvalues and Eigenvectors of the covariance matrix\n",
        "eigen_values , eigen_vectors = np.linalg.eigh(data.cov())\n",
        "print(\"Eigen Values of Covariance Matrix :\",eigen_values)\n",
        "print(\"Eigen Vectors of Covariance Matrix :\")\n",
        "print(eigen_vectors)\n",
        "\n",
        "print(\"\\nStep 5. Sort Eigenvalues in descending order\")\n",
        "# Sort the Eigenvalues in the descending order along with their corresponding Eigenvector.\n",
        "# Remember each column in the Eigen vector-matrix corresponds to a principal component,\n",
        "# So arranging them in descending order of their Eigenvalue \n",
        "# will automatically arrange the principal component in descending order of their variability.\n",
        "# Hence the first column in our rearranged Eigen vector-matrix will be a principal component that captures the highest variability.\n",
        "\n",
        "#sort the eigenvalues in descending order\n",
        "sorted_index = np.argsort(eigen_values)[::-1]\n",
        "print(\"Sorting Index based on Eigen Value :\",sorted_index)\n",
        "sorted_eigenvalue = eigen_values[sorted_index]\n",
        "print(\"Sorting Eigen Value :\",sorted_eigenvalue)\n",
        "#similarly sort the eigenvectors \n",
        "sorted_eigenvectors = eigen_vectors[:,sorted_index]\n",
        "print(\"Sorted Eigen Vectors :\")\n",
        "print(sorted_eigenvectors)\n",
        "\n",
        "print(\"\\nStep 6. Select a subset from the rearranged Eigenvalue matrix\")\n",
        "# Select a subset from the rearranged Eigenvalue matrix as per our need i.e. number_comp = 2. \n",
        "# This means we selected the first two principal components.\n",
        "# select the first n eigenvectors, n is desired dimension of our final reduced data.\n",
        "n_components = 2 #you can select any number of components.\n",
        "print(\"Number of Principal Components :\",n_components)\n",
        "eigenvector_subset = sorted_eigenvectors[:,0:n_components]\n",
        "print(\"Filter Eigen Vector Subset :\")\n",
        "print(eigenvector_subset)\n",
        "# n_components = 2 means our final data should be reduced to just 2 variables. \n",
        "# if we change it to 3 then we get our data reduced to 3 variables.\n",
        "\n",
        "print(\"\\nStep 7. Transform the data : Mean Centering the data  \")\n",
        "# Finally, transform the data by having a dot product between the Transpose of the Eigenvector subset \n",
        "# and the Transpose of the mean-centered data. \n",
        "# By transposing the outcome of the dot product, \n",
        "# the result we get is the data reduced to lower dimensions from higher dimensions.\n",
        "\n",
        "X_mean = data - data.mean()\n",
        "print(\"i. Input Data :\")\n",
        "print(data)\n",
        "print(\"\\nii. Mean of Each Column\")\n",
        "print(data.mean())\n",
        "print(\"\\niii. Mean(Data) = Mean(column) - Column_Value\")\n",
        "print(X_mean)\n",
        "print(\"In above dataframe, I’ve subtracted the mean of each column from each cell of respective column itself.\")\n",
        "print(\"So the mean of each column now is zero.\")\n",
        "\n",
        "# Compute PC1 for row 1. \n",
        "#Transform the data \n",
        "X_reduced = np.dot(eigenvector_subset.transpose(),X_mean.transpose()).transpose()\n",
        "print(\"\\n\")\n",
        "print(\"PCA of Data = Dot Product of Mean(Data) and Eigen Vector Subset\")\n",
        "print(X_reduced)\n",
        "print(\"\\nThe final dimensions of Data reduced to (5,2) and originally the data was of higher dimensions (5,3)\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shape of Data: (5, 3)\n",
            "   X    Y    Z\n",
            "0  1  123  531\n",
            "1  2  142  522\n",
            "2  3  163  573\n",
            "3  4  174  514\n",
            "4  5  195  595\n",
            "\n",
            "Step 1. Calculate Mean of Each Column\n",
            "Mean of X : 3.0\n",
            "Mean of Y : 159.4\n",
            "Mean of Z : 547.0\n",
            "\n",
            "Step 2. Calculate Variance\n",
            "X       2.5\n",
            "Y     780.3\n",
            "Z    1237.5\n",
            "dtype: float64\n",
            "\n",
            "Step 3. Covariance Matrix : (3, 3)\n",
            "      X      Y       Z\n",
            "X   2.5   44.0    30.0\n",
            "Y  44.0  780.3   584.5\n",
            "Z  30.0  584.5  1237.5\n",
            "\n",
            "Step 4. Eigenvalues and Eigenvectors for the calculated Covariance matrix.\n",
            "Eigen Values of Covariance Matrix : [7.92511994e-03 3.82275407e+02 1.63801667e+03]\n",
            "Eigen Vectors of Covariance Matrix :\n",
            "[[ 0.99824773  0.05082148  0.03030913]\n",
            " [-0.05905775  0.82370297  0.56393759]\n",
            " [ 0.00369442 -0.56473941  0.82526102]]\n",
            "\n",
            "Step 5. Sort Eigenvalues in descending order\n",
            "Sorting Index based on Eigen Value : [2 1 0]\n",
            "Sorting Eigen Value : [1.63801667e+03 3.82275407e+02 7.92511994e-03]\n",
            "Sorted Eigen Vectors :\n",
            "[[ 0.03030913  0.05082148  0.99824773]\n",
            " [ 0.56393759  0.82370297 -0.05905775]\n",
            " [ 0.82526102 -0.56473941  0.00369442]]\n",
            "\n",
            "Step 6. Select a subset from the rearranged Eigenvalue matrix\n",
            "Number of Principal Components : 2\n",
            "Filter Eigen Vector Subset :\n",
            "[[ 0.03030913  0.05082148]\n",
            " [ 0.56393759  0.82370297]\n",
            " [ 0.82526102 -0.56473941]]\n",
            "\n",
            "Step 7. Transform the data : Mean Centering the data  \n",
            "i. Input Data :\n",
            "   X    Y    Z\n",
            "0  1  123  531\n",
            "1  2  142  522\n",
            "2  3  163  573\n",
            "3  4  174  514\n",
            "4  5  195  595\n",
            "\n",
            "ii. Mean of Each Column\n",
            "X      3.0\n",
            "Y    159.4\n",
            "Z    547.0\n",
            "dtype: float64\n",
            "\n",
            "iii. Mean(Data) = Mean(column) - Column_Value\n",
            "     X     Y     Z\n",
            "0 -2.0 -36.4 -16.0\n",
            "1 -1.0 -17.4 -25.0\n",
            "2  0.0   3.6  26.0\n",
            "3  1.0  14.6 -33.0\n",
            "4  2.0  35.6  48.0\n",
            "In above dataframe, I’ve subtracted the mean of each column from each cell of respective column itself.\n",
            "So the mean of each column now is zero.\n",
            "\n",
            "\n",
            "PCA of Data = Dot Product of Mean(Data) and Eigen Vector Subset\n",
            "[[-33.79212281 -21.04860048]\n",
            " [-30.4743487   -0.26476795]\n",
            " [ 23.48696187 -11.7178939 ]\n",
            " [-18.96981579  30.71328524]\n",
            " [ 59.74932542   2.31797709]]\n",
            "\n",
            "The final dimensions of Data reduced to (5,2) and originally the data was of higher dimensions (5,3)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JOocEiQOBGh1",
        "outputId": "9afd1f68-aa79-4e4e-e200-e602ca5be702"
      },
      "source": [
        "# PCA in Python Code\n",
        "# I initialize the PCA() class and call the fit_transform() on Data to simultaneously compute the weights of the Principal components \n",
        "# then transform X to produce the new set of Principal components of Data. \n",
        "# This I am storing in the df_pca object, which is converted to a pandas DataFrame.\n",
        "\n",
        "pca = PCA(n_components=2)\n",
        "df_pca = pca.fit_transform(X=data)\n",
        "\n",
        "# Store as dataframe and print\n",
        "df_pca = pd.DataFrame(df_pca)\n",
        "print(\"Original Data Shape :\",data.shape)\n",
        "print(data)\n",
        "print(\"\\n\")\n",
        "print(\"PCA Data Shape :\",df_pca.shape)\n",
        "print(df_pca.round(2))\n",
        "# The first column is the first PC and so on. This dataframe (df_pca) has the same dimensions as the original data X.\n",
        "\n",
        "# Principal Components Weights (Eigenvectors)\n",
        "print(\"\\n\")\n",
        "print(\"Eigen Vectors :\")\n",
        "print(pca.components_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original Data Shape : (5, 3)\n",
            "   X    Y    Z\n",
            "0  1  123  531\n",
            "1  2  142  522\n",
            "2  3  163  573\n",
            "3  4  174  514\n",
            "4  5  195  595\n",
            "\n",
            "\n",
            "PCA Data Shape : (5, 2)\n",
            "       0      1\n",
            "0 -33.79 -21.05\n",
            "1 -30.47  -0.26\n",
            "2  23.49 -11.72\n",
            "3 -18.97  30.71\n",
            "4  59.75   2.32\n",
            "\n",
            "\n",
            "Eigen Vectors :\n",
            "[[ 0.03030913  0.56393759  0.82526102]\n",
            " [ 0.05082148  0.82370297 -0.56473941]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RFRdTo3yCiWE",
        "outputId": "dd1971bf-0e35-4dae-affc-4cdcf8e2e825"
      },
      "source": [
        "# How many components to take ?\n",
        "pca = PCA()\n",
        "df_pca = pca.fit_transform(X=data)\n",
        "# Ratio of Variance explained by each component\n",
        "# We can now look at the proportion of variance explained by each PC.\n",
        "var = pca.explained_variance_ratio_\n",
        "print(var)\n",
        "print(\"So variance explained by each component are 0.81, 0.18, 0.000003\")\n",
        "\n",
        "# Inference : \n",
        "# From the output we find that PC1 explains 81% of the variance, \n",
        "# PC2 explains 18% and so on. \n",
        "# We find that the first 2 components explain approximately 90% of the variance \n",
        "# (0.81+0.18+0.000003)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[8.10778928e-01 1.89217150e-01 3.92274411e-06]\n",
            "So variance explained by each component are 0.81, 0.18, 0.000003\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 325
        },
        "id": "kTyQDoxaJ2Sj",
        "outputId": "b8669f79-4efd-4618-d1b9-db4097b366c1"
      },
      "source": [
        "# PCA CHART\n",
        "\n",
        "# In the above step, we got the proportion of variance explained by each component \n",
        "# which we need to decide the number of components. \n",
        "# We calculated that the first seven components explain most of the variance, \n",
        "# however, we plot the explained variance on a line graph. \n",
        "# Here we plot the ratio of variance explained by each component using a line graph. \n",
        "# This PCA chart helps us to decide the number of principal components to be taken for the modeling algorithm.\n",
        "\n",
        "cumulative_var = np.cumsum(np.round(var, decimals=4)*100)\n",
        "plt.plot(cumulative_var,'k-o',markerfacecolor='None',markeredgecolor='k')\n",
        "plt.title('Principal Component Analysis',fontsize=12)\n",
        "plt.xlabel(\"Principal Component\",fontsize=12)\n",
        "plt.ylabel(\"Cumulative Proportion of Variance Explained\",fontsize=12)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Cumulative Proportion of Variance Explained')"
            ]
          },
          "metadata": {},
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEjCAYAAADzIzwpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5hU1f3H8fcHUJEmoIgUKcaGYAEWYiIW7FGjRhYUxSBSxCiCoihipdjzM0YRQlMB6WhsEREkEBAhYEWxRIqIiAiCoCIC398f964O45a7uzN7t3xfzzPP3rn1s7Ozc+bcc+85MjOcc865gigXdwDnnHMllxcizjnnCswLEeeccwXmhYhzzrkC80LEOedcgXkh4pxzrsC8EHHOOVdgXog455wrsAo5LZC0BsjzTkQza5DSRM4550qMHAsRoFPCdCugM/B3YDXQELgWGJu+aM4554o7Ren2RNIy4CwzW5swrz4ww8yapTGfc865Yixqm0hdYFvSvG1AvdTGcc45V5JELUSeB56XdIakJpLOBJ4N5zvnnCujop7OqgjcBbQnqJV8AUwF7jazH9IZ0DnnXPEVqRBxzjnnshP5PpHwVNZoSS+EzzMknZq+aM4554q7SIWIpF7AMOAT4KRw9g/A4DTlcqWApG2SDinkPm6VNCpFeUzSoanYl0s9SasknV7IfRT6PefyJ2pNpA9wupndB+wO530IHJGWVK5YCv/Jfwj/UddLelJSlZzWN7MqZraiMMc0s3vMrFth9hGVpLMkzZO0VdIGSXMlnV8Uxy4Oon6IS2osabekYUWRKz9S8Z5z+RO1EKkKrAmnsxpR9gJ2pDyRK+7+aGZVgBZABnBb8gqScruJtViSlElwschYoD5QG7gD+GOcuYqpPwPfABdL2ifuMC5eUQuRecAtSfOuA+akNo4rKcIbT18GmsHPp4qukfQJwWnPPU4fhbWWoZJeCr/pL5L0m6z9SWoq6VVJm8Jazq3h/LskjQ+nG4X77CHpC0nrJN2YsI/WkhZK2hwue0zS3nn9LpIE/B8wyMxGmdkWM9ttZnPNrHu4TjlJt0laLekrSWMl7ZeUq4ukNZK+kdRTUitJ74Z5Hks43hWSFoT5tkj6UNJpCcvrSno+fC3+J6l7wrK7JE0Jj79V0vuSMpK2nR7WpFZKui7KtpLGAQ2AF8KaZr9cXqs/E3x5+ImkQjZ8HXpK+iT8vYeG2yDpN5Jek7RR0teSnpZUPZtjHCTpe0n7J8xrEf5Oe0k6NKwlbgn3Mznp+FnvuXMkfRD+rmsT3ysuhcwszwdQB1gCrCJ443wUPj8oyvb+KB2P8O9/ejh9MPA+wQcvBDXUV4GawL4J8w4Np58ENgKtCbrbeRqYFC6rCqwD+gIVw+e/DZfdBYwPpxuF+5wIVAaOBjYkZGoJHB/uvxGwHOiTkP/nPEm/15Hhssa5/O5XAv8DDgGqAM8A45JyDQ/znwlsB/4JHEhwU+5XwMnh+lcAO4HrCWr0FwNbgJrh8nnA4+G+jgt/x1MTXo/twDlAeeBe4I1wWTlgKUENau8w6wqC3iZy3Tb575vL63Ai8CNQA3gUeCFpuQEvAtUJCqUNwNnhskOBM4B9gFrh7/m3HN5f/wKuTlj2MPBoOD0RGBD+vhWBNtn9jQneUyeG0zWAFnH/D5XGR34+QAT8luBekeOBcnGH90fRPsJ/8m3AZoI+1B5nzwLj1KT1kwuRUQnLzgE+DKc7Am/lcMy7+HUhcmTC8geA0Tls2wd4Nrs8SeudEC6rmMvvPhv4S8LzIwi+UFVIyFUvYflG4OKE59MJCzSCQuQLwkvsw3mLgcsJCuddQNWEZfcCTya8HrMSlh0F/BBO/xb4LCl3f+CJvLZN+PvmVYiMAv4ZTv8ufA0OTHqNEz/UpwC35LCvCxP/7uxZiFwMLAinywNfAq3D52OBEUD9bPaZ+J77DLgKqBb3/05pfkS+xNcCi8J/hsUQVPGjbu9KjQvNrLqZNTSzv9ieN5uuyXGrwJcJ098TfKOH4IPz03xkSDzOaoIbYJF0uKQXJX0p6VvgHuCACPvbGP6sk8s6dcNjJR63AkHbSZb1CdM/ZPM88SKEtRZ+0iX9HnWBTWa2NWlZYhdDya9jxbAdqiFQNzyNtFnSZuDWpIw5bZsnSfsSfIl8GsDMFhJ8UF+atGq2f2dJtSVNCk8tfQuMJ+e/z3PAUZIaE9RetpjZ4nBZP4IvtYvDU3JX5rCPdgRfVlaHp79+F+X3dPkT9RLfFuG55u8Ivnn8RFAd/ymd4VyJU9A7V9cQnHqJ6uCE6QYE3+ohuAz9Q+AwM6tG8AGqCPv7KMzQLpd1viD4kE487k72LCjyo15WW0HC/r4IHzUlVU1atpa8rQFWhoV81qOqmZ0TMVNef78/AdWAx8OC+kuCwq1zxP3fEx7j6PDv04kc/j5mtp2gFtOJoIY2LmHZl2bW3czqEtQ0Hlc2l26b2X/N7AKCU4r/DPfnUixqTeIpgkb0DIJ/9kOAxuTvH9+5nLwI1JHUR9I+kqpK+m0u698uqZKkpkAXIKthtSrwLbBN0pHA1VEOHtYIbgj320VStbAhvY2kEeFqE4HrFVzeWoXgA3Gyme3M/68LBB9s14UNxe2BJsC/zGwN8Dpwr6SKko4BuhJ8a8/LYmCrpJsl7SupvKRmklpFzLSe3P+nOwNjCNqijgsfJwDHSjo6wv6rEpwO3SKpHnBTHuuPJTj1dz4JhYik9gp6EYfgKjHjl1sPstbZW9JlkvYzs58I3hd7rONSI2oh0hAYYGbLzWx14iOd4VzZEJ66OYPgSp8vCa7uapvLJnMJGrlnAw+Z2cxw/o0Ep1a2AiP5pXCJkmEawXn4KwlqA+sJbqZ9LlxlDMEH2TxgJUEDda+o+8/GIuAw4GtgCJBpZlmn1ToStLN8QdDR6Z1mNivC77ALOI/gw31luO9RwH4RM90L3BaeCtvjSqbwQ/80gobwLxMeS4EZRKuN3E1wafgW4CWCixNy+30WEHzwv5n0WdMKWCRpG0EnsL0t+3tDLgdWhafOegKXRcjo8ilqB4xPARPM7JX0R3Iue5IaEXw47lWIGkDsJF0BdDOzNnFnKe4kvUbw2ZOSXgtc6kW9Kawi8Kyk+ezZaIaZ/TnlqZxzZV54Gq4FcEHcWVzOohYiH4QP55xLu/Dsx4UEp6q25rW+i493Be+cc67AcqyJSDrJzOaF0zl2+W5mr6UjmHPOueIvx5qIpGVmltUv0soctjczK1GX+R5wwAHWqFGjuGM451yJsXTp0q/NrFZ2y3KsiWQVIOF043QEi0OjRo1YsmRJ3DGcc67EkJTj7RzebYlzzrkCi9pnTjWCzttOJujr5ueuCsysQVqSOeecK/ai1kQeJ7heeyBBV9+9CDpeezhNuZxzzpUAUe8TORNoYmYbJe0ys+ckLQFewAsS55wrs6LWRMoR9HcDQed2+xEM+PKrnjOdc86VHVELkXcI2kMA/kNwemsY8HGUjSWNUTCk6LKEeTUVDIf6SfizRjhfkv6uYFjQdyW1yGGfLSW9F67396RutZ0rMSZOnEizZs0oX748zZo1Y+LEiXFHcqVI2t9fUUauIuge+jfh9IEEPYNOBo6KuP1JBG0qyxLmPUA44hnB+O33h9PnEIzdLYIRFBflsM/F4XKF6/8hSpaWLVuac8XFhAkTrHHjxvbaa6/Zjh077LXXXrPGjRvbhAkT4o7mSoFUvb+AJZbDZ2qRdXsS9sD6ov1yA+NHwClmtk5SHeDfZnaEpH+E0xOT10vYVx1gjpkdGT7vGK5zVV45MjIyzO8TccVFs2bNePTRRylXrhyzZ88GYNWqVcyYMYOePXvGnM6VdMOHD+fss8+mUaNGVKlShX79+jFnzhx69erFsmXL8t5BSNJSM8vIbllu3Z7kNOTkHsxsTOQke6qdUDB8yS9DeNZjz+FPPw/nrUuYVy+cn7xOtiT1AHoANGjgVyS74mP58uXsu+++nHzyyezYsQNJWTVtBg8eHHM6V9KZGePGjUMStWvXpl+/frRp04bly5en7Bi5XZ11eYTtjWCwnkIxM5OUtiqRmY0ARkBQE0nXcZzLr8MPP5wLL7yQgw46iDfffJP999+/QN8UnctOVk23bdtfxnibP38+TZo0Sdkxcuv2JLeR5VJhvaQ6Caezvgrnr2XPMbTr8+vxpdeG83Nbx7libffu3eyzzz6sX7+eoUOHUq1aNebMmUPXrl0ZMmRI3PFcKTBgwAC6du3K6NGjadOmDfPnz0/9+yunxpLkB1CdYHjJm8Kf1aNuG27fiD0b1h9kz4b1B8Lpc9mzYX1xDvtLblg/J0oOb1h3xcWgQYMMsC5duljTpk2tXLly1rRpU29Udyk1YcKEQr+/KGzDetgV/DPAR8BqoAFwJNDOzGZH2H4icApBlynrgTuBfwJTwn2tBjqY2abwUt3HgLOB74EuZrYk3M/bZnZcOJ0BPAnsGxYivSzCL+MN6644mDVrFmeeeSYdO3Zk/Pjx+BXqrjjLrWE9aiHyAXCXmU1JmNceGGThFVIlhRciLm6ff/45zZs358ADD2Tx4sVUrlw57kjO5Sq3QiTqzYZ1gelJ854FDipMMOfKmh07dtChQwe2b9/O9OnTvQBxJV7UQmQccE3SvKuBsamN41zp1q9fPxYuXMiYMWM48sgSVYl3LltRO2BsDvSU1I/gKqh6BHeuL5I0L2slMzsp9RGdKx2mTJnCI488Qu/evWnfvn3ccZxLiaiFyMjw4ZwrgA8//JCuXbvyu9/9jgceeCDuOM6lTKRCxMyeSncQ50qrbdu20a5dOypWrMiUKVPYe++9447kXMpEahORNEpSpaR5dSTNSE8s50oHM+Oqq65i+fLlTJw4kfr16+e9kXMlSNSG9SrAu5J+ByDpEuBd4K10BXOuNBg2bBgTJkxg4MCBnH766XHHcS7lop7OukTSZcBzYa+6dYA/mdn8tKZzrgRbvHgxffr04ZxzzuHWW2+NO45zaRG1JgLBVVnbCcYWWQn8Ly2JnCsFNm7cSPv27albty7jxo2jXLn8/Ks5V3JEbRN5CJgE9CboA+ttgtNbfp2ic0l2795Np06d+PLLL5k2bRo1a9aMO5JzaRP1Et8mwLFmtj58fpOkF4CngKlpSeZcCTV48GBmzJjBsGHDyMjItqcI50qNqG0i52Yzb56kY1IfybmSa+bMmdx111106tSJq67Kc6BN50q8XE9nSbox6fkZSavcnfJEzpVQa9as4dJLL6Vp06YMHz7ce+Z1ZUJebSJ3JD2fnPS8WwqzOFdi7dixg/bt27Njxw7vWNGVKXmdzkr+KpXXc+fKpBtvvJFFixYxdepUDj/88LjjOFdk8qqJJA82ktdz58qcSZMm8eijj3L99deTmZkZdxznilSeNRFJjfmlxlEu6bnXRFyZ9sEHH9CtWzdOOOEE7r///rjjOFfk8ipEKhPcVJhYWHyaMF3omoik3kD38BgjzexvkiYDR4SrVAc2Zw2Lm7TtKmArsAvYmdPIW86lw7Zt28jMzKRy5cpMnjyZvfbaK+5IzhW5XAsRM0vrbbaSmhEUIK2BHcAMSS+a2cUJ6/wV2JLLbtqa2dfpzOlcMjOje/fufPTRR7z66qvUq1cv7kjOxSLuvhiaAIvM7Hsz2wnMBS7KWqjgGskOwMSY8jmXraFDhzJp0iQGDRrEqaeeGncc52ITdyGyDDhR0v5hV/PnAAcnLD8RWG9mn+SwvQEzJS2V1COng0jqIWmJpCUbNmxIWXhXNr3xxhvccMMNnHfeedxyyy1xx3EuVjKL9wIrSV2BvwDfAe8DP5pZn3DZMOB/ZvbXHLatZ2ZrJR0IvAr0MrN52a2bJSMjw5YsWZLS38GVHV9//TUtWrSgfPnyvPnmm9SoUSPuSM6lnaSlObU5x10TwcxGm1nLcHz2b4CPASRVIDi1lXyDY+K2a8OfXwHPErStOJcWu3bt4rLLLmP9+vVMmzbNCxDnyEchImkvSSdKujh8XllSoW/LDWsRSGpAUGhMCBedDnxoZp/nsF1lSVWzpoEzCU6POZcWgwYNYubMmTz66KO0bNky7jjOFQuROmCUdDTwPPAjUJ+gdnAy0Bm4OJdNo5guaX/gJ+AaM9sczr+EpAZ1SXWBUWZ2DlAbeDbsn6gCMMHMfLhelxYzZsxg4MCBdO7cme7du8cdx7liI1KbiKT5wD/MbJykb8ysRvjt/2MzK1HXNnqbiMuvzz77jObNm1O/fn0WLlxIpUqV4o7kXJFKRZtIU2B8OG0AZvYdsG/h4zlXfP3444+0b9+enTt3Mm3aNC9AnEsStRBZBexxElhSa3yIXFfK9e3bl8WLF/PEE09w2GGHxR3HuWIn6siGtwMvSRoO7C2pP9CT4G5z50qlCRMmMHToUPr27ctFF12U9wbOlUGRaiJm9iJwNlCL4K7yhsBFZjYzjdmci837779P9+7dadOmDffee2/ccZwrtqLWRDCztwhuCnSuVNu6dSvt2rWjatWq3rGic3mIVBOR9IykE5PmnShpWnpiORcPM6Nbt2588sknTJo0ibp168YdybliLWrD+snA60nzFgJtUxvHuXg9+uijTJkyhSFDhnDKKafEHce5Yi9qIbKdYGyRRFUIbhB0rlRYuHAhffv25Y9//CP9+vWLO45zJULUQuQV4B+SqgGEPx8D/A5xVyps2LCBDh06cPDBB/PUU09Rrlzs3co5VyJE/U/pC1QDNkn6CtgE7Af0SVcw54rKrl27uPTSS9mwYQPTp0/3jhWdy4dIV2eZ2TfAuZIOIhjvY42ZfZnWZM4VkbvvvptZs2YxatQomjdvHncc50qU/NbZdwMbgUqSDpF0SBoyOVdkXn75ZQYNGkSXLl3o2rVr3HGcK3Gi9uJ7NjAaqJO0yIDyqQ7lXFFYvXo1nTp14thjj2Xo0KFxx3GuRIpaExkKDAIqm1m5hIcXIK5E+vHHH8nMzPy5Y8V99/W+RJ0riKh3rNcg6Ao+3rF0nUuRPn36sGTJEp599lkOPfTQuOM4V2JFrYmMBrqkM4hzRWX8+PEMHz6cm266iQsvvDDuOM6VaFEHpfoPwfjlq4E9rsoKx0YvMXxQqrJt2bJltG7dmlatWjF79mwqVIjcfZxzZVZug1JF/Q8aFT5STlJvgi7lBYw0s79JuiuctyFc7VYz+1c2254NPELQuD/KzO5LR0ZXOnz77be0a9eOatWqMWnSJC9AnEuBqPeJPJWOg0tqRlBYtAZ2ADMkvRguftjMHspl2/IEDf5nAJ8D/5X0vJl9kI6srmQzM7p27cqnn37K7NmzqVMn+UJD51xBRP4qJqk2wYf9AQS1BgDMbEwhjt8EWGRm34fHmAtEHf2nNfA/M1sRbjsJuADwQsT9yiOPPMK0adO4//77Ofnkk+OO41ypEbUr+AuBT4GBwD+AXuHPywt5/GXAiZL2l1QJOIfgjniAayW9K2mMpOz6oagHrEl4/nk4L7v8PSQtkbRkw4YN2a3iSrEFCxZw0003ccEFF3DTTTfFHce5UiXq1VmDgS5m1hz4LvzZA1hamIOb2XLgfmAmQWeObwO7gGHAb4DjgHXAXwt5nBFmlmFmGbVq1SrMrlwJ89VXX9GhQwcaNmzIk08+iaS8N3LORRa1EGlgZlOT5j0F/LmwAcxstJm1DK/y+gb42MzWm9kuM9sNjCQ4dZVsLb/UWgDqh/OcA37pWHHTpk1MmzaN6tWrxx3JuVInaiHyVdgmArBK0u8IagqFvmNd0oHhzwYE7SETJCW2ev6J4LRXsv8Ch0lqLGlv4BLg+cLmcaXHnXfeyezZs3n88cc57rjj4o7jXKkUtWF9JNAGmA48DMwh6IyxUKeZQtMl7U8wwNU1ZrZZ0qOSjiPom2sVcBWApLoEl/KeY2Y7JV1LMNZJeWCMmb2fgjyuFHjppZcYMmQIXbt2pUsXv0/WuXSJdLPhrzYKag2VwzaNEsVvNiz9Vq1aRYsWLWjYsCGvv/6694vlXCGl4mbDPZjZZ4WL5Fx6bN++nczMTHbv3s306dO9AHEuzXIsRCQtN7Mm4fQaglNLv2JmDdKUzbl86927N0uXLuW5557jkEN8uBvn0i23mkj3hOlO6Q7iXGGNHTuWESNGcPPNN3P++efHHce5MiHHQsTM5sPP3YtcCfQwsx+LKphz+fHee+/Rs2dPTjnlFAYPHhx3HOfKjDwv8TWzXcCZBFdjOVfsbNmyhXbt2lG9enUmTpzoHSs6V4Si3ifyMHC3pL3SGca5/DIzrrzySlasWMHkyZM56KCD4o7kXJkS9StbL+Ag4AZJG0hoZPeGdRenhx9+mGeeeYYHH3yQE088Me44zpU5UQsRb1h3xc78+fPp168ff/rTn+jbt2/ccZwrk6KOJzI33UGcy4/169fToUMHGjduzBNPPOEdKzoXk/yMJ3IccCK/Hk/kjjTkci5HO3fupGPHjmzevJkZM2aw3377xR3JuTIr6ngiPYAFwKnAzcDRQF/g0PRFcy57d9xxB3PmzGHYsGEcc8wxccdxrkyLenVWP+BsM/sT8EP4M5Og00TniswLL7zAvffeS/fu3encuXPccZwr8yJ1wCjpWzOrFk5vBGqZ2W5Jm8ysZrpDppJ3wFhyrVixgpYtW3LIIYewYMECKlasGHck58qEVHTA+LmkRma2CvgYuEDS18COFGV0LldZHSsCTJs2zQsQ54qJqIXIA0ATgrE9BgLTgL2B69ITy7k99erVi7feeosXXniBxo0bxx3HORfKtRCRNAV4EhgbDlWLmb0sqQawt5ltS39EV9Y9+eSTjBo1iv79+3PeeefFHcc5lyCvhvW1wGjgC0n/J+kYADPb4QWIKwrvvPMOV199NW3btmXgwIFxx3HOJcm1EDGz64F6BL34HgQslPSOpBsSxlwvFEm9JS2T9L6kPuG8ByV9KOldSc9Kqp7DtqskvSfpbUneWl7KbNmyhczMTGrUqOEdKzpXTEXpxXe3mf3LzC4F6gCPAOcBqyW9WJiDS2pGMG5Ja+BY4DxJhwKvAs3M7BiChvz+ueymrZkdl9OVA65kMjOuuOIKVq5cyZQpU6hdOyXfWZxzKRb1PhEAzOxb4F/Ay8B64KRCHr8JsMjMvjezncBc4CIzmxk+B3gDqF/I47gS5q9//Sv//Oc/efDBB2nTpk3ccZxzOYh6x3pFSZdKegVYDZwF3E5QMymMZcCJkvaXVAk4Bzg4aZ0rCQqt7BgwU9LS8K56VwrMmzePW265hczMTPr06RN3HOdcLvK6OusU4M9AO2AdMA7obmafpeLgZrZc0v3ATOA74G1gV8LxBwA7gadz2EUbM1sr6UDgVUkfmtm8bH6PHkAPgAYNvOf64uzLL7/k4osv5je/+Q2jR4/2jhWdK+byqok8C/xI0OXJkWY2JFUFSBYzG21mLc3sJOAbgjYQJF1B0PZymeVwW72ZrQ1/fhVmbZ3DeiPMLMPMMmrVqpXK+C6Fdu7cySWXXMKWLVuYNm0a1apVizuScy4PeV3uclC6x1WXdKCZfSWpAXARcLykswn66zrZzL7PYbvKQDkz2xpOn0lwI6QroW677Tbmzp3L2LFjOfroo+OO45yLINdCJN0FSGi6pP0JOnO8xsw2S3oM2IfgFBXAG2bWU1JdYJSZnQPUBp4Nl1cAJpjZjCLI69Lg+eef5/777+eqq67i8ssvjzuOcy6iSB0wlibeAWPx8+mnn9KyZUsOPfRQ5s+f7/1iOVfM5NYBY45tImHXJs6l1Q8//EC7du0oV66cd6zoXAmUW8P66qwJSbOKIIsrg6699lreeecdxo0bR6NGjeKO45zLp9wKke8lNZNUHmitQLnkR1EFdaXPmDFjGDNmDAMGDODcc8+NO45zrgBya1i/G1hM0MANwf0aiURws1/5NORypdzbb7/NNddcw2mnncbdd98ddxznXAHlWIiY2TBJIwk6XvwQaMovBYdzBbZ582batWvH/vvvz4QJEyhf3r+HOFdS5XWJ706CUQ2bm9nq3NZ1LoqsjhU/++wz5s6dy4EHHhh3JOdcIURt01gl6W5JKyVtl7QifL53WtO5UufBBx/kueee46GHHuL3v/993HGcc4WUn+FxWwNXEVy11ZCgA8ZqwPXpieZKm7lz59K/f386dOjAddf5yMrOlQaRbjaU9DlwrJltTJh3APCOmdVLY76U85sN47Fu3TqaN29O9erV+e9//0vVqlXjjuSciyi3mw2j1kRy6krVu1h1ecrqWHHr1q3Mnj3bCxDnSpGobSJTgRcknSWpSdhB4j+BKemL5kqLW2+9lXnz5jFixAiaNm0adxznXApFrYn0A24DhgJ1gbXAJGBwmnK5UuLZZ5/lwQcf5Oqrr+ayyy6LO45zLsW8A0aXNp988gkZGRkcccQR/Oc//2GfffbJeyPnXLFToA4YnSuM77//nszMTCpUqMDUqVO9AHGulIp6Osu5yMyMa665hvfee4+XXnqJhg0bxh3JOZcmXhNxKTd69GiefPJJbrvtNv7whz/EHcc5l0ZeiLiUevPNN7n22ms544wzuPPOO+OO45xLs8insySdCRwHVEmcb2Z3FCaApN5Ad4J7Tkaa2d8k1QQmA42AVUAHM/smm207E1w1BjDYzJ4qTBZXON988w2ZmZnUqlWLp59+2jtWdK4MiFQTCcc8Hw+0BA5OeNQvzMElNSMoQFoDxwLnSToUuAWYbWaHAbPD58nb1gTuBH4bbn+nj8YYn927d9O5c2c+//xzpk6dSq1ateKO5JwrAlFrIpcSdHuyJsXHbwIsMrPvASTNBS4CLgBOCdd5Cvg3cHPStmcBr5rZpnDbV4GzgYkpzugieOCBB3jhhRf4+9//zvHHHx93HOdcEYnaJvI1sDkNx18GnChpf0mVgHMIaji1zWxduM6XQO1stq0HJBZqn4fzXBGbM2cOAwYM4JJLLuHaa6+NO45zrghFrYn8FXha0r3A+sQFZraioAc3s+WS7gdmAt8BbwO7ktYxSYW6I1JSD6AHQIMGDQqzK5fkiy++4JJLLuHwww9n5MiRSN6dmnNlSdSayDDgPGAB8L+ExyeFDWBmo82spZmdBHwDfAysl1QHIPz5VTabrm2GiqQAAB45SURBVCWotWSpH87L7hgjzCzDzDL8XH3q/PTTT1x88cV89913TJ8+nSpVquS9kXOuVIlUiJhZuRwehb78RtKB4c8GBO0hE4Dngc7hKp2B57LZ9BXgTEk1wgb1M8N5roj079+f+fPnM3LkSI466qi44zjnYpCvO9bDD/p6wOcpbGSfLml/4CfgGjPbLOk+YIqkrgSDYHUIj58B9DSzbma2SdIg4L/hfgZmNbK79Js+fTp//etfueaaa+jYsWPccZxzMYk6KFUdgl57fwdsBPYH3gAuMbMv0powxbwDxsL7+OOPycjIoEmTJsybN8/7xXKulEtFB4zDgHeAGmZWB6gBvAUMT01EV1Jkday49957e8eKzrnIp7PaAHXM7CcAM/tOUj9yaMh2pZOZcfXVV7Ns2TJefvllv9LNORe5JvINkNxyegTpuXfEFVMjR45k7Nix3HHHHZx11llxx3HOFQNRayIPALMkjSZo6G4IdAFuT1cwV7wsXbqUXr16cdZZZ3H77f5nd84FIhUiZjZS0qcE3Z8cA3wBXGpms9MZzhUPmzZtIjMzk9q1azN+/HjvWNE597PIl/ia2WvAa2nM4oqh3bt38+c//5m1a9cyf/58DjjggLgjOeeKkRwLEUkDzGxIOD0wp/UK2xW8K97uu+8+XnrpJR577DFat24ddxznXDGTW00ksZv3g3Ncy5Vas2fP5vbbb6djx4785S9/iTuOc64YinSzYWniNxtGs3btWpo3b84BBxzA4sWLvV8s58qwQt9sKCnb7kQkZdcxoivhsjpW/P77771jRedcrqI2rO+VPEPSXoBfplMK3XzzzSxYsIBJkybRpEmTuOM454qxXAsRSf8BDKgoaV7S4vrA6+kK5uIxdepUHn74YXr16sXFF18cdxznXDGXV01kFCCgFTA6Yb4RDE7ll/yWIh999BFXXnklxx9/PA899FDccZxzJUCuhYiZPSWpPHAuMMnMfiyaWK6offfdd7Rr146KFSsyZcoU9t5777gjOedKgDzbRMxsl6S2BON9uFLIzOjZsycffPABr7zyCgcf7Fd0O+eiidoB41igZzqDuPj84x//YPz48dx9992cccYZccdxzpUgUQelmg/8lqDr9zUEbSIAhGOjlxh+n8ielixZwgknnMBpp53Giy++SLlyUb9XOOfKitzuE4l6ie/I8JFykq4HuhEUTO8R9A78KlA1XOVAYLGZXZjNtrvCbQA+M7Pz05GxtNq4cSOZmZkcdNBBjBs3zgsQ51y+Re3F96l0HFxSPeA64Cgz+0HSFIIhd09MWGc68FwOu/jBzI5LR7bSbvfu3Vx++eWsW7eO+fPns//++8cdyTlXAkX+6impi6TXJH0U/uySogwVgH0lVQAqEXQzn3XMasCpwD9TdCwXuueee3j55Zf529/+RqtWreKO45wroaJ2ezIAuAWYRFBzmAT0C+cXmJmtBR4CPgPWAVvMbGbCKhcCs83s2xx2UVHSEklvSPrV6S6XvVmzZnHHHXdw2WWX0bOnXy/hnCu4qA3rK4FTzGx1wryGwDwza1jgg0s1gOnAxQRD7U4FppnZ+HD5y8AoM5uew/b1zGytpEMIbnw8zcw+zWa9HkAPgAYNGrRcvXp18iplxueff07z5s2pXbs2ixYtonLlynFHcs4Vc4XugBGoDGxImrcR2LcwwYDTgZVmtsHMfgKeAX4PIOkAoDXwUk4bhzUZzGwF8G+geQ7rjTCzDDPLqFWrViEjl1w7duygQ4cObN++nenTp3sB4pwrtKiFyAzgaUlHSNpX0pHAU8ArhTz+Z8DxkipJEnAasDxclgm8aGbbs9tQUg1J+4TTBwAnAB8UMk+pdtNNN7Fw4ULGjBnDEUccEXcc51wpELUQuRbYCrwLfAe8E/7sVZiDm9kiYBrwJsGluuWAEeHiS4CJietLypA0KnzaBFgi6R1gDnCfmXkhkoPJkyfz97//nd69e9O+ffu44zjnSol8DUolqRxwAPC1me1OW6o0Kos3G3744Ye0atWKo48+mn//+9/eL5ZzLl9ScbMhkg4DOgB1gS8kTTGzT1KU0aXJtm3bvGNF51zaRL3E91LgLeAYgtNYRwNvhvNdMWVmXHXVVXz44YdMmjSJ+vXrxx3JOVfKRK2JDAbOMbOfB6aSdCIwDpiQjmCu8IYNG8aECRMYPHgwp512WtxxnHOlUNSG9arAwqR5bxBc+uuKocWLF9OnTx/OPfdc+vfvH3cc51wpFbUQ+T/gHkkVASTtCwwJ57tiZuPGjbRv35569eoxduxY71jROZc2UU9n/QU4COgt6RugBsGwueskXZ21kpk1SH1Elx+7d++mU6dOfPnllyxYsICaNWvGHck5V4pFLUQ6pTWFS5nBgwczY8YMhg8fTkZGtlfkOedcykTtCn5uuoO4wps5cyZ33XUXl19+OT169Ig7jnOuDIh6ie9eku6WtELS9vDn3ZL8poNiYs2aNVx66aU0bdqU4cOHE/Qi45xz6RX1dNYDBJ0h9gRWAw2B24FqwPXpieai2rFjB+3bt2fHjh1Mnz6dSpUqxR3JOVdGRC1E2gPHmtnG8PlHkt4k6EPLC5GY9e3bl0WLFjF16lQOP/zwuOM458qQqNd+5nRuxM+ZxGzSpEk89thjXH/99WRmZsYdxzlXxkQtRKYCL0g6S1ITSWcTDFk7JX3RXF4++OADunXrxgknnMD9998fdxznXBkU9XRWP+A2YChBB4xrCYbIHZymXC4P27ZtIzMzk8qVKzN58mT22muvuCM558qgPAsRSeWBkUAPM7sj/ZFcXsyM7t2789FHHzFr1izq1asXdyTnXBmVZyFiZrsknQmUyPFDSqOhQ4cyadIk7rnnHtq2bRt3HOdcGRa1TeRhwO8LKQbeeOMNbrjhBs477zxuvvnmuOM458q4SCMbSlpD0HfWLmAD8PNGJa2/rJI8suHXX39NixYtqFChAkuXLqVGjRpxR3LOlQGpGNkwbX1nSboe6EZQML0HdAGGAycDW8LVrjCzt7PZtjNBgz/AYDN7Kl0547Zr1y4uu+wyvvrqK15//XUvQJxzxUKUhvUzgWbAW2Y2J5UHl1QPuA44ysx+kDQFuCRcfJOZTctl25rAnUAGQQG0VNLzZvZNKjMWF4MGDWLmzJmMGDGCFi1axB3HOeeAPNpEJN0MPEvwwf6CpGvSkKECsK+kCkAl4IuI250FvGpmm8KC41Xg7DTki92MGTMYOHAgnTt3plu3bnHHcc65n+XVsH4VcJqZtSb40P5LKg9uZmuBh4DPgHXAFjObGS4eIuldSQ9L2iebzesBaxKefx7O+xVJPSQtkbRkw4YNKfwN0m/16tVcdtllHH300Tz++OPesaJzrljJqxA5wMzeADCzBQSN6ykjqQZwAdCY4CbGypI6Af2BI4FWQE2gUJchmdkIM8sws4xatWoVMnXR+fHHH2nfvj07d+5k2rRp3rGic67YyfMSXwXKhTcdKuF5OUmFHXf1dGClmW0ws5+AZ4Dfm9k6C/wIPEHQg3CytcDBCc/rh/NKjRtuuIH//ve/PPHEExx22GFxx3HOuV/JqxCoAuwEfgJ2ANUTnmf9LIzPgOMlVVJwnuY0YLmkOhCUWMCFwLJstn0FOFNSjbBGc2Y4r1SYMGECjz/+OH379uWiiy6KO45zzmUrr6uzGqfz4Ga2SNI04E2CQuktYATwsqRaBL0Ev00wjgmSMoCeZtbNzDZJGgT8N9zdQDPblM68ReX999+ne/funHjiidx7771xx3HOuRxFutmwNCnuNxtu3bqVVq1asXnzZt566y3q1KkTdyTnXBmXipsNXREwM7p168Ynn3zC7NmzvQBxzhV7XogUI48++ihTpkzhvvvu45RTTok7jnPO5amwV1e5FFm4cCF9+/bl/PPPp1+/fnHHcc65SPJViEg6WNLx6QpTVm3YsIEOHTrQoEEDnnrqKb+h0DlXYkQ6nSWpATAROI6gn6oqkjKBs83M++EohF27dnHppZeyYcMGFi5cSPXq1eOO5JxzkUWtifwDeAmoyi/3hrwKnJGOUGXJ3XffzaxZsxg6dCjNmzePO45zzuVL1Ib11sC5ZrZbkgGY2RZJ+6UvWun38ssvM2jQILp06ULXrl3jjuOcc/kWtSayHjg0cYakowjuOHcFsGrVKjp16sSxxx7L0KFD447jnHMFErUQeQh4UVIXoIKkjsBk4P60JSvFkjtW3HfffeOO5JxzBRLpdJaZjZG0kaBr+DXAn4Hbzeyf6QxXWvXp04clS5bw7LPPcuihh+a9gXPOFVNRr84qb2bPAc+lOU+pN378eIYPH06/fv248MIL447jnHOFEvV01peSHpd0QlrTlHLLli2jR48enHzyyQwZMiTuOM45V2hRC5EzgW3AREkrJd0r6eg05ip1vv32W9q1a8d+++3HpEmTqFDBe5xxzpV8kQoRM3vLzPqZWQPgCqAG8Jqkd9MZrrQwM7p27cqnn37K5MmTOeiglA4Q6ZxzsSnI1+EPgeUEl/f6cHsRPPLII0ybNo0HHniAk046Ke44zjmXMpFqIpKqS+oqaTawAjiF4PLeA9OYrVRYsGABN910ExdeeCE33nhj3HGccy6lotZEvgBeByYA7cxsc/oilR5fffUVHTp0oGHDhjzxxBPesaJzrtSJWoj8xszWpSOApOuBbgQdO74HdAFGAxkE/XQtBq4ys1+N5y5pV7gNwGdmdn46MhZEVseKmzZt4o033vCOFZ1zpVKOhYikk8xsXvi0iaQm2a1nZq8V9OCS6gHXAUeZ2Q+SpgCXAE8DncLVJhAUMsOy2cUPZnZcQY+fTnfeeSezZ89mzJgxHHvssXHHcc65tMitJvI40CycHp3DOgYckoIM+0r6CagEfGFmM7MWSloM1C/kMYrUSy+9xJAhQ+jatStdunSJO45zzqWNzCzeAFJvYAjwAzDTzC5LWLYXsAjobWb/yWbbncDbwE7gvpy6YZHUA+gB0KBBg5arV69O+e+RZeXKlbRo0YJGjRrx+uuve79YzrkST9JSM8vIblnUq7Oy7e5E0jOFDFYDuABoDNQFKkvqlLDK48C87AqQUMPwF7sU+Juk32S3kpmNMLMMM8uoVatWYSLnavv27WRmZmJmTJ8+3QsQ51ypF/WO9bY5zD+lkMc/HVhpZhvChvNngN8DSLoTqAXckNPGZrY2/LkC+DcQ66hOvXv35s0332Ts2LEcckhhz/I551zxl+vVWZIGhpN7J0xnOQQo7Hmhz4DjJVUiOJ11GrBEUjfgLOA0M9udQ7YawPdm9qOkA4ATgAcKmafAxo4dy4gRI7jllls4//xic5GYc86lVV6X+B4c/iyXMA1Bg/oa4K7CHNzMFkmaBrxJ0K7xFjAC+I6ggFoY3lvxjJkNlJQB9AzHdW8C/EPS7jDffWb2QWHyFNR7771Hz549adu2LYMGDYojgnPOxSJSw7qk7mY2sgjypF1GRoYtWbIkZfvbsmULrVq1Ytu2bbz11lvUrl07Zft2zrniILeG9aiDUo0Md1QVOABQwrIVqQhZEpkZV155JStWrGDOnDlegDjnypyog1I1Ibjp71iCU1kKfwKUT0+04u/hhx/mmWee4aGHHuLEE0+MO45zzhW5qFdnDQPmADWBbwm6gv8H0DlNuYq9+fPn069fPy666CJuuCHHC8icc65Ui9om8g1woJn9JGmzmVWXVBlYZmaN054yhVLRJrJ+/XqaN29O5cqVWbJkCfvtt1+K0jnnXPFT6DYRYDuwF0GHiF9LagB8A+yfmoglx86dO+nYsSObN29mxowZXoA458q0qKez/gN0CKenAS8Dc4ECd75YUt1xxx3MmTOHYcOGccwxx8QdxznnYhX16qwOCU9vBd4HqgBj0xGquJk4cSJDhgxh+fLl7N69m7Zt29K5c5ltDnLOuZ9FrYn8zMx2m9k4MxtmZt+lI1RxMnHiRAYMGED//v2pUqUKhx12GCtXrmTixIlxR3POudjlNp7IOH65jDdHZvbnlCYqZoYMGcKwYcPo378/5cqV45VXXmHVqlX06tWLjh07xh3POedildvprP8VWYpibPny5Rx//PE0a9aMgQMH0rhxY+rXr8/y5cvjjuacc7HLsRAxs7uLMkhx1aRJk5975s0yf/58mjTJdqBH55wrU6LesX5qTssKMzxuSTBgwAC6du3K6NGjadOmDfPnz6dr164MGTIk7mjOORe7qPeJJA+PWwvYG/icwg+PW6xltXv06tWL5cuX06RJE4YMGeLtIc45RwGHx5VUHrgN2Gpm/5fyVGmU6l58nXOutCv08LjJzGwXwbjo/QoTzDnnXMlWoEIkdAaQ7aiDzjnnyoaoDetr2POekUpAReAv6QjlnHOuZIjasN4p6fl3wMdm9m2K8zjnnCtBovadNTfdQZxzzpU8UccT2Q+4DmhO0PHiz8zszPRESw9JG4DVBdz8AODrFMZJFc+VP54rfzxX/pTGXA3NrFZ2C6KezppKMAzus8APBQxRLOT0QkQhaUlOl7nFyXPlj+fKH8+VP2UtV9RC5HjgADPbkeoAzjnnSq6ol/jOB45MZxDnnHMlT9SayBXAvyQtAtYnLjCzgakOVYyNiDtADjxX/niu/PFc+VOmckVtWB8JnE8wTG5im4iV9vFEnHPO5SxqIbIVONzM1qU/knPOuZIiapvICuCndAaJk6SzJX0k6X+Sbslm+T6SJofLF0lqlLCsfzj/I0lnFXGuGyR9IOldSbMlNUxYtkvS2+Hj+SLOdYWkDQnH75awrLOkT8JHSgeqj5Dr4YRMH0vanLAsna/XGElfSVqWw3JJ+nuY+11JLRKWpfP1yivXZWGe9yS9LunYhGWrwvlvS0ppj6YRcp0iaUvC3+uOhGW5vgfSnOumhEzLwvdUzXBZOl+vgyXNCT8L3pfUO5t10vceM7M8H8CNwOtAR+DUxEeU7Yvzg+DS5U8JurTfG3gHOCppnb8Aw8PpS4DJ4fRR4fr7AI3D/ZQvwlxtgUrh9NVZucLn22J8va4AHstm25oEX0hqAjXC6RpFlStp/V7AmHS/XuG+TwJaAMtyWH4O8DIggishF6X79YqY6/dZxwP+kJUrfL6K4IrNOF6vU4AXC/seSHWupHX/CLxWRK9XHaBFOF0V+Dib/8m0vcei1kSuCYPeQzC2SNZjVMTti7PWwP/MbIUFlzBPAi5IWucC4KlwehpwmiSF8yeZ2Y9mtpJgSOHWRZXLzOaY2ffh0zeA+ik6dqFy5eIs4FUz22Rm3wCvAmfHlKsjMDFFx86Vmc0DNuWyygXAWAu8AVSXVIf0vl555jKz18PjQtG9v6K8XjkpzHsz1bmK8v21zszeDKe3AsuBekmrpe09FqkQMbPGOTxKw4BU9YA1Cc8/59d/gJ/XMbOdwBZg/4jbpjNXoq4E3zSyVJS0RNIbki5MUab85GoXVpunSTo4n9umMxfhab/GQOKonOl6vaLIKXs6X6/8Sn5/GTBT0lJJPWLI8ztJ70h6WVLTcF6xeL0kVSL4IJ6eMLtIXi8Fp9qbA4uSFqXtPRb1El9XjEnqBGQAJyfMbmhmayUdArwm6T0z+7SIIr0ATDSzHyVdRVCLy3GI5RhcAkyzYFycLHG+XsWapLYEhUibhNltwtfrQOBVSR+G39SLwpsEf69tks4B/gkcVkTHjuKPwAIzS6y1pP31klSFoODqY0XYOW6kmoikNZI+y+6R7oBFYC1wcMLz+uG8bNeRVAHYD9gYcdt05kLS6cAA4Hwz+zFrvpmtDX+uAP5N8O2kSHKZ2caELKOAllG3TWeuBJeQdKohja9XFDllT+frFYmkYwj+hheY2cas+Qmv11cE3SGl6jRunszsWzPbFk7/C9hL0gEUg9crlNv7Ky2vl6S9CAqQp83smWxWSd97LGLDzclJj0uAhUDvdDQUFeWDoDa2guD0RlZjXNOkda5hz4b1KeF0U/ZsWF9B6hrWo+RqTtCQeFjS/BrAPuH0AcAnpKiBMWKuOgnTfwLesF8a8VaG+WqE0zWLKle43pEEjZwqitcr4RiNyLmh+Fz2bPRcnO7XK2KuBgTtfL9Pml8ZqJow/TpwdhHmOijr70fwYfxZ+NpFeg+kK1e4fD+CdpPKRfV6hb/7WOBvuayTtvdYYYIfBLydyj9QXA+CKxc+JvhAHhDOG0jw7R6CAbimhv9Qi4FDErYdEG73EfCHIs41i6AHgbfDx/Ph/N8D74X/RO8BXYs4173A++Hx5wBHJmx7Zfg6/g/oUpS5wud3AfclbZfu12sisI7gMvnPCU4N9QR6hssFDA1zvwdkFNHrlVeuUcA3Ce+vJeH8Q8LX6p3w7zygiHNdm/D+eoOEQi6790BR5QrXuYLgYpvE7dL9erUhaHN5N+FvdU5Rvcci3WyYHUk1gFVmtl+BduCcc67Eizo8bnL/WJX45bpj55xzZVTUq7MOTnr+HfB/wLjUxnHOOVeSFPh0lnPOOZfrJb6STpB0fw7L7pN0fHpiOeecKwnyuk/kViCnG2L+TXBlknPOuTIqr0LkOGBGDstm8ctNZM7FStK28G7zwuzjVkkp6Q9Okkk6NBX7cq44y6sQqUZw00529iLoMdK5lAu7zv4hLBzWS3oy7NYhW2ZWxYK7zQvMzO4xs255r1l4ks6SNE/SVgXd5s+VdH5RHLs4CP++p8edwxVeXoXIh8CZOSw7M1zuXLr80cyqEHS/nQHclrxC2A1NiSIpk+Dm1bEE3UzUBu4g6HPJuRIlr0LkYeAfki6SVA5AUjlJFwHDCS7zdS6tLOh36GWgGfx8qugaSZ8QdFGyx+mjsNYyVNJL4Tf9RZJ+k7U/SU0lvSppU1jLuTWcf5ek8eF0o3CfPSR9IWmdpBsT9tFa0kJJm8Nlj0nKqdZOwnYi+L8ZZGajzGyLme02s7lm1j1cp5yk2yStVjAI0lhJ+yXl6hL2afeNpJ6SWoW9Jm+W9FjC8a6QtCDMt0XSh5JOS1heV9Lz4WvxP0ndE5bdJWlKePytCgY8ykjadnpYk1op6boo20oaR9ClygthTbNfhLeBK64i3FJ/A7AV2AF8Ef7cCtyQylv3/eGPxAdB/1anh9MHE3QXMSh8bgTjHtQE9k2Yd2g4/SRBB5mtCe6FepqwKwqCU7DrgL4E3dlUBX4bLrsLGB9ONwr3OZGgv6OjgQ0JmVoS9EFUIVx3OUHvqSTnSfq9jgyXNc7ld8/qhuIQoArwDDAuKdfwMP+ZwHaCnmwPJOjG+yvg5HD9K4CdwPUEp6AvJhjKoGa4fB7weLiv48Lf8dSE12M7wY3F5Qm6s8nqB60csJSgBrV3mHUFcFZe2yb/ff1Rsh/RVgraRs4CLg1/Vos7uD9K9yP8kNkGbAZWhx90iQXGqUnrJxcioxKWnQN8GE53BN7K4ZjZFSKJ/X49AIzOYds+wLPZ5Ula74RwWcVcfvfZwF8Snh9B0F9ThYRc9RKWbwQuTnie1R14ViHyBXt2OLkYuJygcN5F2DlguOxe4MmE12NWwrKjgB/C6d8CnyXl7g88kde2CX9fL0RKwSPS+WQL+qZ/Jcq6zqXQhWY2K4dla3KYn+XLhOnvCb7RQ/DBmZ9xQhKPs5qgRoKkwwlOS2UQdANUgeCbeV6yulOvQ9BjanbqhsdKPG4FgraTLOsTpn/I5nniRQhrLfzkTthf3fCxyYLR8BKXZSQ8T34dK4btUA2BukoYp56gxvGfvLa1YGA3V0pEHR7XueKmoF0trCE49RJVYpc/DQi+1QMMI7iw5DAzq0ZwT5Ui7O+jMEO7XNb5guBDOvG4O9mzoMiPemFbTOL+vggfNSVVTVoWZTyJNcBKM6ue8KhqZudEzORdZZQSXoi4suZFoI6kPpL2kVRV0m9zWf92SZUUDMHaBZgczq8KfAtsk3QkcHWUg4c1ghvC/XaRVC1sSG8jaUS42kTgekmNw8ua7wEmF+Ib/IHAdZL2ktQeaAL8y8zWEIxtca+kigoGoOoKjI+wz8XAVkk3S9pXUnlJzSS1iphpPfkrzF0x5YWIK1PCUzdnEFxO+yXB1V1tc9lkLkEj92zgITObGc6/kaCNcCswkl8KlygZphE0cF9JUBtYDwwGngtXGUPQuek8glNe24FeUfefjUUEw8d+DQwBMu2XUQo7ErSzfEEw4t6duZxCTPwddgHnETTGrwz3PYpgUKYo7gVuC68muzHPtV2x5R0wOpcNSY0IPhz3Ksnn8CVdAXQzszZ5retcQXhNxDnnXIF5IeKcc67A/HSWc865AvOaiHPOuQLzQsQ551yBeSHinHOuwLwQcc45V2BeiDjnnCswL0Scc84V2P8D9MoXNMU8H7YAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MjEcGz0bK7wX",
        "outputId": "e28a18c4-8206-4809-ef8a-c0458416ba24"
      },
      "source": [
        "# How to get the original features back\n",
        "# The fitted pca object has the inverse_transform() method that gives back the original data when you input principal components features.\n",
        "print(\"PCA Dataframe\")\n",
        "print(df_pca.round(2))\n",
        "df_orig = pca.inverse_transform(df_pca)\n",
        "print(\"\\n\")\n",
        "print(\"Original Dataframe :\")\n",
        "print(pd.DataFrame(df_orig).round())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "PCA Dataframe\n",
            "[[-3.379e+01 -2.105e+01 -9.000e-02]\n",
            " [-3.047e+01 -2.600e-01  6.000e-02]\n",
            " [ 2.349e+01 -1.172e+01  1.200e-01]\n",
            " [-1.897e+01  3.071e+01 -1.000e-02]\n",
            " [ 5.975e+01  2.320e+00 -7.000e-02]]\n",
            "\n",
            "\n",
            "Original Dataframe :\n",
            "     0      1      2\n",
            "0  1.0  123.0  531.0\n",
            "1  2.0  142.0  522.0\n",
            "2  3.0  163.0  573.0\n",
            "3  4.0  174.0  514.0\n",
            "4  5.0  195.0  595.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GeeXl2GrhyB_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "3b913077-7fb7-4c71-9f7d-da83b2d228a0"
      },
      "source": [
        "# Let's Take another example\n",
        "# Load the Datasets \n",
        "\n",
        "# Boston Dataset\n",
        "boston = datasets.load_boston()\n",
        "boston_pd = pd.DataFrame(boston.data)\n",
        "boston_pd.columns = boston.feature_names\n",
        "boston_pd[\"MEDV\"] = boston.target\n",
        "boston_pd.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.00632</td>\n",
              "      <td>18.0</td>\n",
              "      <td>2.31</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.538</td>\n",
              "      <td>6.575</td>\n",
              "      <td>65.2</td>\n",
              "      <td>4.0900</td>\n",
              "      <td>1.0</td>\n",
              "      <td>296.0</td>\n",
              "      <td>15.3</td>\n",
              "      <td>396.90</td>\n",
              "      <td>4.98</td>\n",
              "      <td>24.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.02731</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>6.421</td>\n",
              "      <td>78.9</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>396.90</td>\n",
              "      <td>9.14</td>\n",
              "      <td>21.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.02729</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>7.185</td>\n",
              "      <td>61.1</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>392.83</td>\n",
              "      <td>4.03</td>\n",
              "      <td>34.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.03237</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>6.998</td>\n",
              "      <td>45.8</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>394.63</td>\n",
              "      <td>2.94</td>\n",
              "      <td>33.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.06905</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>7.147</td>\n",
              "      <td>54.2</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>396.90</td>\n",
              "      <td>5.33</td>\n",
              "      <td>36.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      CRIM    ZN  INDUS  CHAS    NOX  ...    TAX  PTRATIO       B  LSTAT  MEDV\n",
              "0  0.00632  18.0   2.31   0.0  0.538  ...  296.0     15.3  396.90   4.98  24.0\n",
              "1  0.02731   0.0   7.07   0.0  0.469  ...  242.0     17.8  396.90   9.14  21.6\n",
              "2  0.02729   0.0   7.07   0.0  0.469  ...  242.0     17.8  392.83   4.03  34.7\n",
              "3  0.03237   0.0   2.18   0.0  0.458  ...  222.0     18.7  394.63   2.94  33.4\n",
              "4  0.06905   0.0   2.18   0.0  0.458  ...  222.0     18.7  396.90   5.33  36.2\n",
              "\n",
              "[5 rows x 14 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QUY-zIK_h3KN",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "4d992342-5866-4de9-d0ea-088cbc3a5496"
      },
      "source": [
        "# Removing Response  variable\n",
        "\n",
        "# As PCA works in an unsupervised learning setup, \n",
        "# therefore we will remove the dependent i.e. response variable from our dataset. \n",
        "# Note that PCA only works on numeric variables, \n",
        "# and that is why we create dummy variables for categorical variables. \n",
        "# As here we have only one categorical variable ‘Chas’ which is a binary categorical variable, \n",
        "# we don’t require creating dummy variable and can use all the independent variables for performing PCA.\n",
        "\n",
        "BosData2 = boston_pd[['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', \n",
        "                    'TAX','PTRATIO', 'B', 'LSTAT']]\n",
        "BosData2.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.00632</td>\n",
              "      <td>18.0</td>\n",
              "      <td>2.31</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.538</td>\n",
              "      <td>6.575</td>\n",
              "      <td>65.2</td>\n",
              "      <td>4.0900</td>\n",
              "      <td>1.0</td>\n",
              "      <td>296.0</td>\n",
              "      <td>15.3</td>\n",
              "      <td>396.90</td>\n",
              "      <td>4.98</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.02731</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>6.421</td>\n",
              "      <td>78.9</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>396.90</td>\n",
              "      <td>9.14</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.02729</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>7.185</td>\n",
              "      <td>61.1</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>392.83</td>\n",
              "      <td>4.03</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.03237</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>6.998</td>\n",
              "      <td>45.8</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>394.63</td>\n",
              "      <td>2.94</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.06905</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>7.147</td>\n",
              "      <td>54.2</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>396.90</td>\n",
              "      <td>5.33</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      CRIM    ZN  INDUS  CHAS    NOX  ...  RAD    TAX  PTRATIO       B  LSTAT\n",
              "0  0.00632  18.0   2.31   0.0  0.538  ...  1.0  296.0     15.3  396.90   4.98\n",
              "1  0.02731   0.0   7.07   0.0  0.469  ...  2.0  242.0     17.8  396.90   9.14\n",
              "2  0.02729   0.0   7.07   0.0  0.469  ...  2.0  242.0     17.8  392.83   4.03\n",
              "3  0.03237   0.0   2.18   0.0  0.458  ...  3.0  222.0     18.7  394.63   2.94\n",
              "4  0.06905   0.0   2.18   0.0  0.458  ...  3.0  222.0     18.7  396.90   5.33\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iVK6uCriiMvJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b3bd2eb3-9740-4c83-c73b-bf92b1ce7615"
      },
      "source": [
        "# Scaling Features\n",
        "\n",
        "# We will have to first scale the dataset to perform PCA in Python.\n",
        "scale = StandardScaler()\n",
        "scaled_data = scale.fit_transform(BosData2)\n",
        "scaled_data"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.41978194,  0.28482986, -1.2879095 , ..., -1.45900038,\n",
              "         0.44105193, -1.0755623 ],\n",
              "       [-0.41733926, -0.48772236, -0.59338101, ..., -0.30309415,\n",
              "         0.44105193, -0.49243937],\n",
              "       [-0.41734159, -0.48772236, -0.59338101, ..., -0.30309415,\n",
              "         0.39642699, -1.2087274 ],\n",
              "       ...,\n",
              "       [-0.41344658, -0.48772236,  0.11573841, ...,  1.17646583,\n",
              "         0.44105193, -0.98304761],\n",
              "       [-0.40776407, -0.48772236,  0.11573841, ...,  1.17646583,\n",
              "         0.4032249 , -0.86530163],\n",
              "       [-0.41500016, -0.48772236,  0.11573841, ...,  1.17646583,\n",
              "         0.44105193, -0.66905833]])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bMCJtZthiogJ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "dbc6de61-912a-44e0-d086-ae9b94cb8279"
      },
      "source": [
        "# We can change the above output into a dataset.\n",
        "\n",
        "scaled_data = pd.DataFrame(scaled_data,columns=BosData2.columns)\n",
        "scaled_data.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.419782</td>\n",
              "      <td>0.284830</td>\n",
              "      <td>-1.287909</td>\n",
              "      <td>-0.272599</td>\n",
              "      <td>-0.144217</td>\n",
              "      <td>0.413672</td>\n",
              "      <td>-0.120013</td>\n",
              "      <td>0.140214</td>\n",
              "      <td>-0.982843</td>\n",
              "      <td>-0.666608</td>\n",
              "      <td>-1.459000</td>\n",
              "      <td>0.441052</td>\n",
              "      <td>-1.075562</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.417339</td>\n",
              "      <td>-0.487722</td>\n",
              "      <td>-0.593381</td>\n",
              "      <td>-0.272599</td>\n",
              "      <td>-0.740262</td>\n",
              "      <td>0.194274</td>\n",
              "      <td>0.367166</td>\n",
              "      <td>0.557160</td>\n",
              "      <td>-0.867883</td>\n",
              "      <td>-0.987329</td>\n",
              "      <td>-0.303094</td>\n",
              "      <td>0.441052</td>\n",
              "      <td>-0.492439</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-0.417342</td>\n",
              "      <td>-0.487722</td>\n",
              "      <td>-0.593381</td>\n",
              "      <td>-0.272599</td>\n",
              "      <td>-0.740262</td>\n",
              "      <td>1.282714</td>\n",
              "      <td>-0.265812</td>\n",
              "      <td>0.557160</td>\n",
              "      <td>-0.867883</td>\n",
              "      <td>-0.987329</td>\n",
              "      <td>-0.303094</td>\n",
              "      <td>0.396427</td>\n",
              "      <td>-1.208727</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.416750</td>\n",
              "      <td>-0.487722</td>\n",
              "      <td>-1.306878</td>\n",
              "      <td>-0.272599</td>\n",
              "      <td>-0.835284</td>\n",
              "      <td>1.016303</td>\n",
              "      <td>-0.809889</td>\n",
              "      <td>1.077737</td>\n",
              "      <td>-0.752922</td>\n",
              "      <td>-1.106115</td>\n",
              "      <td>0.113032</td>\n",
              "      <td>0.416163</td>\n",
              "      <td>-1.361517</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-0.412482</td>\n",
              "      <td>-0.487722</td>\n",
              "      <td>-1.306878</td>\n",
              "      <td>-0.272599</td>\n",
              "      <td>-0.835284</td>\n",
              "      <td>1.228577</td>\n",
              "      <td>-0.511180</td>\n",
              "      <td>1.077737</td>\n",
              "      <td>-0.752922</td>\n",
              "      <td>-1.106115</td>\n",
              "      <td>0.113032</td>\n",
              "      <td>0.441052</td>\n",
              "      <td>-1.026501</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       CRIM        ZN     INDUS  ...   PTRATIO         B     LSTAT\n",
              "0 -0.419782  0.284830 -1.287909  ... -1.459000  0.441052 -1.075562\n",
              "1 -0.417339 -0.487722 -0.593381  ... -0.303094  0.441052 -0.492439\n",
              "2 -0.417342 -0.487722 -0.593381  ... -0.303094  0.396427 -1.208727\n",
              "3 -0.416750 -0.487722 -1.306878  ...  0.113032  0.416163 -1.361517\n",
              "4 -0.412482 -0.487722 -1.306878  ...  0.113032  0.441052 -1.026501\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0x1kHwr4izjq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b8bc9b13-d578-49bd-bef5-c7580d757b34"
      },
      "source": [
        "# Splitting the dataset into train and test\n",
        "\n",
        "# PCA should not be made to run on the entire dataset \n",
        "# As this would cause the dataset to leak thus causing overfitting\n",
        "# Also, we should not perform PCA on train and test separately \n",
        "# as the level of variance will be different in both these datasets \n",
        "# which will the cause the final vectors of these two datasets to have different directions\n",
        "# We first divide the dataset into train and test and perform PCA on train dataset \n",
        "# and transform the test dataset using that PCA model (which was fitted on the train dataset). \n",
        "\n",
        "# Target Variable\n",
        "Y = boston_pd['MEDV']\n",
        "X_train,X_test,Y_train,Y_test=train_test_split(scaled_data,Y,test_size=0.3,random_state=123) \n",
        "\n",
        "# Initialize and Fit PCA\n",
        "# We first initialize PCA for having 13 components (for 13 continuous variables in the dataset) \n",
        "# and then we fit this model on the scaled features.\n",
        "\n",
        "pca = PCA(n_components=13)\n",
        "pca_model = pca.fit(X_train)\n",
        "print(pca_model)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "PCA(copy=True, iterated_power='auto', n_components=13, random_state=None,\n",
            "    svd_solver='auto', tol=0.0, whiten=False)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OaM6FWQtjfun",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c0a74c24-65c3-4f88-db51-9f81202e65f9"
      },
      "source": [
        "# Generate PCA Loadings\n",
        "\n",
        "# We use transform command which transforms the scaled data to PCA loadings for each observation.\n",
        "pca_train = pca_model.transform(X_train)\n",
        "pca_train"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-2.20660682, -1.55051779,  1.69405915, ...,  0.41926072,\n",
              "         0.12959831,  0.00521784],\n",
              "       [-2.89522044,  0.63233154, -0.11783641, ..., -0.16824372,\n",
              "         0.27292552,  0.19829253],\n",
              "       [-1.32856526, -0.86934574, -0.67716436, ...,  0.45628438,\n",
              "        -0.26832397,  0.13759222],\n",
              "       ...,\n",
              "       [-1.14293586,  0.00729923, -1.03490865, ..., -0.12767168,\n",
              "         0.17449274,  0.07324418],\n",
              "       [ 3.67339539,  0.22908568, -0.4579643 , ...,  0.33265531,\n",
              "        -0.09832891,  0.12570271],\n",
              "       [ 3.34152955,  0.53789073, -1.73620654, ..., -0.91230342,\n",
              "        -0.58960347,  0.28672463]])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IEyvrqSCkESK",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 452
        },
        "outputId": "e1ae1fa2-9d1a-4c62-f067-bf38eb92b40f"
      },
      "source": [
        "# Generate Loading Matrix\n",
        "\n",
        "# We now generate the principal components loading matrix \n",
        "# by using the attribute components_ of the pca command for each variable.\n",
        "\n",
        "Variable_Names =['crim','zn','indus','chas','nox','rm','age','dis','rad','tax','ptratio','black','lstat'] \n",
        "Matrix = pd.DataFrame(pca_model.components_,columns=Variable_Names)\n",
        "Matrix1 = np.transpose(Matrix)\n",
        "Matrix1\n",
        "\n",
        "# Inference :\n",
        "# This Loading Matrix is like a correlation matrix. \n",
        "# The variable having the highest correlation with the columns will be the first principal component\n",
        "#  For eg, the variable indus has the highest correlation with PC1, therefore, indus will be PC 1 \n",
        "#  Heading in the output should be PC1, PC2 and so on. Will be renaming them in the upcoming steps"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>crim</th>\n",
              "      <td>0.261028</td>\n",
              "      <td>0.402021</td>\n",
              "      <td>0.272849</td>\n",
              "      <td>0.083141</td>\n",
              "      <td>-0.056245</td>\n",
              "      <td>0.649569</td>\n",
              "      <td>0.433235</td>\n",
              "      <td>0.057887</td>\n",
              "      <td>0.215429</td>\n",
              "      <td>0.063131</td>\n",
              "      <td>-0.128398</td>\n",
              "      <td>0.064637</td>\n",
              "      <td>-0.040974</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>zn</th>\n",
              "      <td>-0.248594</td>\n",
              "      <td>0.378921</td>\n",
              "      <td>0.236975</td>\n",
              "      <td>0.288462</td>\n",
              "      <td>0.135785</td>\n",
              "      <td>0.023437</td>\n",
              "      <td>-0.443978</td>\n",
              "      <td>-0.463129</td>\n",
              "      <td>0.217877</td>\n",
              "      <td>0.354617</td>\n",
              "      <td>0.201162</td>\n",
              "      <td>-0.086564</td>\n",
              "      <td>0.058977</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>indus</th>\n",
              "      <td>0.344874</td>\n",
              "      <td>-0.105353</td>\n",
              "      <td>-0.039082</td>\n",
              "      <td>0.016200</td>\n",
              "      <td>0.008651</td>\n",
              "      <td>-0.099526</td>\n",
              "      <td>-0.308914</td>\n",
              "      <td>0.097501</td>\n",
              "      <td>0.734052</td>\n",
              "      <td>-0.271809</td>\n",
              "      <td>-0.226253</td>\n",
              "      <td>-0.045638</td>\n",
              "      <td>0.295420</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>chas</th>\n",
              "      <td>-0.004040</td>\n",
              "      <td>-0.306065</td>\n",
              "      <td>0.234459</td>\n",
              "      <td>0.740967</td>\n",
              "      <td>-0.490540</td>\n",
              "      <td>-0.166000</td>\n",
              "      <td>0.174522</td>\n",
              "      <td>0.017959</td>\n",
              "      <td>0.020874</td>\n",
              "      <td>-0.013390</td>\n",
              "      <td>0.009374</td>\n",
              "      <td>-0.031098</td>\n",
              "      <td>-0.040853</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>nox</th>\n",
              "      <td>0.340383</td>\n",
              "      <td>-0.191293</td>\n",
              "      <td>0.153285</td>\n",
              "      <td>0.059000</td>\n",
              "      <td>0.137624</td>\n",
              "      <td>0.007177</td>\n",
              "      <td>-0.244216</td>\n",
              "      <td>0.085684</td>\n",
              "      <td>-0.095101</td>\n",
              "      <td>0.281641</td>\n",
              "      <td>0.052457</td>\n",
              "      <td>0.798674</td>\n",
              "      <td>-0.074809</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rm</th>\n",
              "      <td>-0.199428</td>\n",
              "      <td>-0.158725</td>\n",
              "      <td>0.640229</td>\n",
              "      <td>-0.421555</td>\n",
              "      <td>-0.181577</td>\n",
              "      <td>-0.063845</td>\n",
              "      <td>0.073905</td>\n",
              "      <td>-0.289464</td>\n",
              "      <td>0.115754</td>\n",
              "      <td>-0.383456</td>\n",
              "      <td>0.168155</td>\n",
              "      <td>0.174593</td>\n",
              "      <td>-0.044029</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>age</th>\n",
              "      <td>0.317344</td>\n",
              "      <td>-0.336219</td>\n",
              "      <td>0.057659</td>\n",
              "      <td>-0.063865</td>\n",
              "      <td>0.072904</td>\n",
              "      <td>0.098528</td>\n",
              "      <td>0.008672</td>\n",
              "      <td>-0.608906</td>\n",
              "      <td>-0.224804</td>\n",
              "      <td>0.154259</td>\n",
              "      <td>-0.522464</td>\n",
              "      <td>-0.211679</td>\n",
              "      <td>0.029531</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>dis</th>\n",
              "      <td>-0.320117</td>\n",
              "      <td>0.385590</td>\n",
              "      <td>-0.109335</td>\n",
              "      <td>0.173370</td>\n",
              "      <td>-0.051794</td>\n",
              "      <td>-0.110102</td>\n",
              "      <td>-0.067926</td>\n",
              "      <td>-0.104640</td>\n",
              "      <td>-0.137332</td>\n",
              "      <td>-0.381458</td>\n",
              "      <td>-0.596248</td>\n",
              "      <td>0.395582</td>\n",
              "      <td>0.009908</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rad</th>\n",
              "      <td>0.306986</td>\n",
              "      <td>0.286627</td>\n",
              "      <td>0.257125</td>\n",
              "      <td>-0.048092</td>\n",
              "      <td>-0.234509</td>\n",
              "      <td>-0.073091</td>\n",
              "      <td>-0.206277</td>\n",
              "      <td>0.156843</td>\n",
              "      <td>-0.465397</td>\n",
              "      <td>-0.069271</td>\n",
              "      <td>0.068748</td>\n",
              "      <td>-0.124110</td>\n",
              "      <td>0.621425</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>tax</th>\n",
              "      <td>0.337749</td>\n",
              "      <td>0.234736</td>\n",
              "      <td>0.160595</td>\n",
              "      <td>-0.037207</td>\n",
              "      <td>-0.150966</td>\n",
              "      <td>-0.101342</td>\n",
              "      <td>-0.378124</td>\n",
              "      <td>0.140673</td>\n",
              "      <td>-0.110272</td>\n",
              "      <td>-0.148007</td>\n",
              "      <td>-0.072336</td>\n",
              "      <td>-0.242639</td>\n",
              "      <td>-0.713897</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ptratio</th>\n",
              "      <td>0.202096</td>\n",
              "      <td>0.249431</td>\n",
              "      <td>-0.388886</td>\n",
              "      <td>-0.250190</td>\n",
              "      <td>-0.605856</td>\n",
              "      <td>-0.232696</td>\n",
              "      <td>0.154015</td>\n",
              "      <td>-0.334577</td>\n",
              "      <td>0.156726</td>\n",
              "      <td>0.219550</td>\n",
              "      <td>0.134384</td>\n",
              "      <td>0.185844</td>\n",
              "      <td>-0.029675</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>black</th>\n",
              "      <td>-0.192293</td>\n",
              "      <td>-0.249771</td>\n",
              "      <td>-0.251010</td>\n",
              "      <td>-0.030737</td>\n",
              "      <td>-0.395729</td>\n",
              "      <td>0.663866</td>\n",
              "      <td>-0.448709</td>\n",
              "      <td>-0.021258</td>\n",
              "      <td>-0.082559</td>\n",
              "      <td>-0.147828</td>\n",
              "      <td>0.084647</td>\n",
              "      <td>0.038187</td>\n",
              "      <td>0.016798</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>lstat</th>\n",
              "      <td>0.325826</td>\n",
              "      <td>0.068240</td>\n",
              "      <td>-0.241014</td>\n",
              "      <td>0.279321</td>\n",
              "      <td>0.276294</td>\n",
              "      <td>0.060112</td>\n",
              "      <td>0.094466</td>\n",
              "      <td>-0.377758</td>\n",
              "      <td>-0.121975</td>\n",
              "      <td>-0.551158</td>\n",
              "      <td>0.443936</td>\n",
              "      <td>0.073138</td>\n",
              "      <td>-0.020351</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               0         1         2   ...        10        11        12\n",
              "crim     0.261028  0.402021  0.272849  ... -0.128398  0.064637 -0.040974\n",
              "zn      -0.248594  0.378921  0.236975  ...  0.201162 -0.086564  0.058977\n",
              "indus    0.344874 -0.105353 -0.039082  ... -0.226253 -0.045638  0.295420\n",
              "chas    -0.004040 -0.306065  0.234459  ...  0.009374 -0.031098 -0.040853\n",
              "nox      0.340383 -0.191293  0.153285  ...  0.052457  0.798674 -0.074809\n",
              "rm      -0.199428 -0.158725  0.640229  ...  0.168155  0.174593 -0.044029\n",
              "age      0.317344 -0.336219  0.057659  ... -0.522464 -0.211679  0.029531\n",
              "dis     -0.320117  0.385590 -0.109335  ... -0.596248  0.395582  0.009908\n",
              "rad      0.306986  0.286627  0.257125  ...  0.068748 -0.124110  0.621425\n",
              "tax      0.337749  0.234736  0.160595  ... -0.072336 -0.242639 -0.713897\n",
              "ptratio  0.202096  0.249431 -0.388886  ...  0.134384  0.185844 -0.029675\n",
              "black   -0.192293 -0.249771 -0.251010  ...  0.084647  0.038187  0.016798\n",
              "lstat    0.325826  0.068240 -0.241014  ...  0.443936  0.073138 -0.020351\n",
              "\n",
              "[13 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GVL7sLT5kPVT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "286d21af-edc1-4172-faa2-2f7a8bd0e81d"
      },
      "source": [
        "# Variance explained by each Principal Component\n",
        "\n",
        "# We took the number of components for PCA equal to the number of variables in our dataset \n",
        "# which is 13 in our case \n",
        "# we will figure out the optimum value of the number of components to run PCA \n",
        "# i.e. reduce the number of components to be considered for the modeling algorithms \n",
        "# thus in a way reducing the number of features\n",
        "# In order to decide the number of Principal Components, \n",
        "# we analyze the proportion of variance explained by each component\n",
        "# We use the explained_variance function for computing variance explained by each Principal Component.\n",
        "\n",
        "pca_model.explained_variance_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([6.15535877, 1.3837924 , 1.17884792, 0.82071583, 0.7800156 ,\n",
              "       0.66886856, 0.56912498, 0.38614551, 0.27744284, 0.22153714,\n",
              "       0.19341501, 0.15764127, 0.0663099 ])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f7PArB5pk4Jm",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2adb17de-2c8e-4eed-c8b8-ad700b50c1f1"
      },
      "source": [
        "# Ratio of Variance explained by each component\n",
        "\n",
        "# We can now look at the proportion of variance explained by each PC.\n",
        "var = pca_model.explained_variance_ratio_\n",
        "var\n",
        "\n",
        "# Inference : \n",
        "# From the output we find that PC1 explains 47% of the variance, \n",
        "# PC2 explains 11% and so on. \n",
        "# We find that the first seven components explain approximately 90% of the variance \n",
        "# (0.478 + 0.107 + 0.091 + 0.063 + 0.060 + 0.052 + 0.044 = 0.898)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0.47867295, 0.10761095, 0.09167339, 0.06382316, 0.0606581 ,\n",
              "       0.05201472, 0.04425814, 0.0300287 , 0.02157541, 0.01722789,\n",
              "       0.01504097, 0.01225901, 0.00515661])"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KQ35Mm91lA4h",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 325
        },
        "outputId": "1bd33637-a91e-444d-8981-834f1cb867f9"
      },
      "source": [
        "# PCA CHART\n",
        "\n",
        "# In the above step, we got the proportion of variance explained by each component \n",
        "# which we need to decide the number of components. \n",
        "# We calculated that the first seven components explain most of the variance, \n",
        "# however, we plot the explained variance on a line graph. \n",
        "# Here we plot the ratio of variance explained by each component using a line graph. \n",
        "# This PCA chart helps us to decide the number of principal components to be taken for the modeling algorithm.\n",
        "\n",
        "cumulative_var = np.cumsum(np.round(var, decimals=4)*100)\n",
        "plt.plot(cumulative_var,'k-o',markerfacecolor='None',markeredgecolor='k')\n",
        "plt.title('Principal Component Analysis',fontsize=12)\n",
        "plt.xlabel(\"Principal Component\",fontsize=12)\n",
        "plt.ylabel(\"Cumulative Proportion of Variance Explained\",fontsize=12)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Cumulative Proportion of Variance Explained')"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEjCAYAAAAomJYLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxN9f/A8dfbIGshe1/CjwpTxCh8tViSUCrVF1mSLKmR5Bua+rYZqW/x7SvKMtI6JFsqyTKRpEJpm6iUfcm+fA2G9++Pc0bXNMuZmXvn3Jl5Px+P87jnnnPvOe8zy33f8zmf8/6IqmKMMcakVsjvAIwxxoQnSxDGGGPSZAnCGGNMmixBGGOMSZMlCGOMMWmyBGGMMSZNliCMMcakyRKEMcaYNBVOb4WIbAEyvYtOVasHNSJjjDFhId0EAXQPmG8C9AL+C2wCLgTuB14PXWjGGGP8JF5KbYjI98D1qrotYNnfgI9UNTKE8RljjPGJ12sQVYEjqZYdAS4IbjjGGGPChdcE8R7wnohcJyJ1RaQtMMddbowxJh/y2sRUDHgCuB3nbGI7MBN4UlWPhTJAY4wx/vCUIIwxxhQ8nu+DcJuX4kRkvvs8SkRahS40Y4wxfvKUIEQkGngZ+Bm42l18DBgZorhMGBORIyJSK4fbeEREpgQpHhWR2sHYlgk+EfldRNrkcBs5/pszWef1DGIw0EZVRwOn3WU/AReHJCqTq9x/4GPuP+EuEZkmIqXSe72qllLVjTnZp6qOUtV7crINr0TkehFZLiKHReQPEVkmIjflxr7DgdcPaBGpKSKnReTl3IgrK4LxN2eyzmuCKA1scedTLloUAU4EPSLjlxtVtRTQCIgCHk39AhHJ6MbKsCQit+F0qHgd+BtQCfgXcKOfcYWpnsB+4B8ico7fwRj/eU0Qy4HhqZYNAhKCG47xm3sz5AIgEs4039wnIj/jNDGe1aTjnm2MF5EP3G/oX4jI/6VsT0Tqi8giEdnnnp084i5/QkTedOdruNvsJyLbRWSHiAwN2MYVIvK5iBxw170kIkUzOxYREWAM8LSqTlHVg6p6WlWXqWpf9zWFRORREdkkIrtF5HUROS9VXL1FZIuI7BeRASLSRES+deN5KWB/d4nIZ258B0XkJxFpHbC+qoi85/4sfhGRvgHrnhCRd9z9HxaRH0QkKtV7Z7lnQL+JyCAv7xWRN4DqwHz3DPHhDH5WPXG+GJwkVQJ1fw4DRORn97jHu+9BRP5PRJaKyF4R2SMib4lImTT2UVlE/ici5wcsa+QeUxERqe2e3R10tzMj1f5T/ubai8iP7rFuC/xbMUGmqplOQBVgNfA7zh/Pevd5ZS/vtym8J/f32sadrwb8gPOhCs4Z4yKgHFA8YFltd34asBe4Aqd0y1vAdHddaWAH8BBQzH1+pbvuCeBNd76Gu814oCRwKfBHQEyNgabu9msAicDggPjPxJPquC5x19XM4NjvBn4BagGlgNnAG6niesWNvy2QBMwFKuLcKLobuMZ9/V1AMvAgzhn2P4CDQDl3/XJggruthu4xtgr4eSQB7YEI4BlglbuuELAG58ynqBvrRpzqBhm+N/XvN4Ofw1XAcaAsMA6Yn2q9Au8DZXASzh9AO3ddbeA64Byggnuc/0nn7+tD4N6AdWOBce58PBDjHm8xoEVav2Ocv6mr3PmyQCO//4fy65SVDxEBrsS5F6IpUMjv4G0K0h+B8w98BDiAU2trAmcng1apXp86QUwJWNce+Mmd7wp8nc4+n+CvCeKSgPXPAXHpvHcwMCeteFK97u/uumIZHPsSYGDA84txvgQVDojrgoD1e4F/BDyfhZuscBLEdtzu4+6yL4EeOIn3FFA6YN0zwLSAn8figHX1gGPu/JXA5lRxjwBezey9Ab/fzBLEFGCuO9/M/RlUTPUzDvzAfgcYns62bg78vXN2gvgH8Jk7HwHsBK5wn78OTAL+lsY2A//mNgP9gXP9/t/J75Pnbq7q+ML9h/gSnNNzr+83Ye9mVS2jqheq6kA9+wbILem+y7EzYP5/ON/EwflQ/DULMQTuZxPOTZmIyEUi8r6I7BSRQ8AooLyH7e11H6tk8Jqq7r4C91sY51pFil0B88fSeB54QX+bup9iqY6jKrBPVQ+nWhdYrib1z7GYe93nQqCq27RzQEQOAI+kijG992ZKRIrjfPF7C0BVP8f5EO6W6qVp/p5FpJKITHebew4Bb5L+72ceUE9EauKcdRxU1S/ddQ/jfBH90m0muzudbXTG+SKyyW2SaublOE3Wee3m2shtAz6K883iJM6p9MlQBmfCRnbvptyC0xziVbWA+eo438bB6WL9E1BHVc/F+XAUD9tb78bQOYPXbMf5AA7cbzJnJ4GsuCClbT5ge9vdqZyIlE61bhuZ2wL85ibwlKm0qrb3GFNmv79bgHOBCW4S3omTuHp53P4odx+Xur+f7qTz+1HVJJyzj+44Z1ZvBKzbqap9VbUqzhnCBEmj+7KqfqWqnXCa+ea62zMh4PUM4DWcC9JROP/wtYCaZO2f3xQ87wNVRGSwiJwjIqVF5MoMXv+YiJQQkfpAbyDlImVp4BBwREQuAe71snP3m/wQd7u9ReRc96J0CxGZ5L4sHnhQnC6epXA+7GaoanLWDxdwPrQGuRddbwfqAh+q6hZgJfCMiBQTkcuAPjjftjPzJXBYRIaJSHERiRCRSBFp4jGmXWT8v9oLmIpz7aehO/0daCAil3rYfmmcJsqDInIB8M9MXv86TnPcTQQkCBG5XZwq0eD0plL+7Faf8pqiInKniJynqidx/i7Oeo0JHq8J4kIgRlUTVXVT4BTK4Eze5janXIfTI2YnTi+olhm8ZRnOBeMlwPOq+rG7fChOc8dhYDJ/Jg4vMbyL0+59N863+F04N3jOc18yFedDajnwG87F3miv20/DF0AdYA8QC9ymqilNXV1xrmtsxyl2+biqLvZwDKeAjjgf3L+5254CnOcxpmeAR93mqbN6/Lgf6K1xLirvDJjWAB/h7SziSZzu0QeBD3Au9Gd0PJ/hfKivTfUZ0gT4QkSO4BQCfUDTvvehB/C725w1ALjTQ4wmG7wW63sNeFtVF4Y+JFPQiEgNnA++Ijn45u47EbkLuEdVW/gdS7gTkaU4nylBuZvehIbXG5+KAXNEZAVnX6hCVXsGPSpjTL7lNo01Ajr5HYvJmNcE8aM7GWNMtrmtETfjNB8dzuz1xl9W7tsYY0ya0j2DEJGrVXW5O59uWW9VXRqKwIwxxvgr3TMIEfleVVPq8fyWzvtVVcOiq2v58uW1Ro0afodhjDF5ypo1a/aoaoW01qV7BpGSHNz5mqEILJhq1KjB6tWr/Q7DGGPyFBFJ93YFK5VhjDEmTV5rtZyLUxDsGpwaK2duo1fV6iGJzBhjjK+8nkFMwOm3/BRO2edonGJeY0MUlzHGGJ95vQ+iLVBXVfeKyClVnSciq4H5WJIwxph8yesZRCGcOivgFEw7D2fQDhso3hhj8imvCWIdzvUHgE9xmpxeBjZ4ebOITBVnOMfvA5aVE2coyp/dx7LuchGR/4ozJOO3ItLI++EYY0zBER8fT2RkJBEREURGRhIfHx/U7XtNEH1xRoUCeABnkJQyOGPYejENaJdq2XBgiarWwanemTLm9Q041TDrAP1wEpExxpgA8fHxxMTEMG7cOJKSkhg3bhwxMTFBTRKeEoSqblTVX9353ap6j6r+Q1U91Wdy78jel2pxJ5xxJnAfbw5Y/ro7gt0qoIyIZDQimDHGhJVQfLNPSkpi69atfPPNNyxevJiHH36Yjh07smzZMtatW0fLli2Ji4sjNjY2CEfgyKjURnrD/Z1FVadmc9+VVHWHO7+TP4dPvICzh57c6i7bQSoi0g/nLIPq1a23rTHGfynf7OPi4mjRogUrVqygT58+AHTt2hWAEydOsHfvXvbs2ZPplPK6o0eP/mVf48aNA6By5cpERUXRokULEhMTg3YsGfVi6uHh/Yoz4EqOqKqKSJarBqrqJJxBzomKirKqg8YY3z399NMMGTKE9evXM3/+fPbs2UOFChXo06cPMTEx7N27l0OHDqX7/nPPPZfy5ctTvnx5KleuTGRk5JnngdPdd9/Ns88+y0033UThws5H+YoVK6hbt27QjiWjUhsZjfwVDLtEpIqq7nCbkHa7y7dx9tjEf8PbuL3GGJNrVJXNmzezbt26s6ZffvmF6GhnUMISJUpQsWJFypUrx7Fjx2jevHmaH/YpU7ly5ShatKin/T/11FMMHTqUsmXLnnWmkitNTKmJSBmgA1AVZ8jED1T1QA72/R7OcIaj3cd5AcvvF5HpwJXAwYCmKGOMyXVJSUn88MMPf0kGBw78+RH4f//3fzRo0IBDhw4RHR1N9+7dufDCCxEREhISiI6O5s03vQxB7k1Kc1V0dDSJiYnUrVuX2NjYM8uDQlUznYBWwAGc8XbfAVa5z1t7fH88zjWEkzjXFPoA5+P0XvoZWAyUc18rwHjgV+A7IMrLPho3bqzGGJOZt99+W+vXr6+FChXS+vXr69tvv33W+h07duiCBQt09OjR2rVrV61Xr55GREQoTpO6lixZUps2bar9+/fXCRMm6GeffaaHDh06a/s1a9bUpUuX6okTJ3Tp0qVas2bNv+wnXACrNb3P7vRW6Nkf8D8Cd6Radjvwk5f358ZkCcIYk5nAD++jR4/qlClTtEKFCtqhQwe97rrrtGLFimcSAaDVqlXTjh076qOPPqozZ87UDRs26KlTpzztJ6MkFE4yShCeRpQTkQPA+ap6KmBZYWCPqpbJ1qlLkEVFRamV+zbGpEVV2bRpE1dddRVNmjRh8+bNfPfdd5w4cQIAEeHyyy+nQYMGZ6bLLruMcuXK+Rx56InIGlWNSmud12sQbwD3Af8NWHYv8HoOYzPGmKBLSkpi7dq1rFy5ks8//5zPP/+cHTucS5n79u3jyiuvZPDgwTRo0IB69erRuHFj1qxZ43PU4cdrgrgcGCAiD+P0KLoAqAh8ISLLU16kqlcHP0RjjMnYtm3bzkoGa9euPXN2UKtWLVq3bk2zZs0YM2YMr7zyCm3atDnz3oSEhKB2Dc1PvCaIye5kjDG+OnHiBN98882ZZLBy5Uq2bHHurS1WrBhNmjRh8ODBNGvWjGbNmlGpUqUz7y1btiz9+vX7y01swewamp94ShCq+lrmrzLGmJyJj48nNjb2TLfNmJgYWrVqdSYRfP7556xevZqkpCTAqaDQvHlzmjVrRvPmzWnQoEGG9xHkStfQfMTrReopwCBV/V/AsirAq6qaugifL+witTF5W0qJihEjRpCUlMS8efNYtmwZycnJABQtWpRGjRqdSQjNmjXjggsu8DnqvC8YF6lLAd+KSA9V/VxEugDjgCnBCtIYU/CoKj/++CNLly4lJiYGVaVfv34AVKlShWbNmrFhwwZmz55No0aNKFasmM8RFyxem5i6iMidwDwRWQ9UAW5R1RUhjc4Yk6+oKr/++isJCQksXbqUhIQEdu3adWZ97969ad26NS1atKB69eokJydTrFgxmjdv7mPUBZfnUhs4vZeSgFo4N879EpKIjDH5ypYtW84khKVLl565oFy1alWuu+46WrZsScuWLbnxxhvp0aMHLVv+WQYu2MXnTNZ4ShAi8jzQHefeh/eBUThNTvep6swQxmeMyWN27959VkL45Rfnu+T5559Py5YtGTFiBK1ateKiiy5CRM68LyYmhj59+lgPo3CS3i3WgRPwAc74DYHLrgZ+8/L+3Jis1IYxoZNR6Yh9+/bpnDlzNDo6WiMjI8+UqTj33HP1xhtv1LFjx+q6devyXYmK/IKcltpIj4iUVtXDOU9TOWe9mIwJjdQD4CxatIg+ffrQuHFjdu7cydq1a1FVihcvzlVXXUWrVq1o2bIljRo1OjNOgQlfGfViyjBBiMhQVX0+4Pl1qroo4PkYVR0S1GizyRKEMaERGRnJ0KFD2bZtGx9++CFffvklycnJiAhXX301LVu2pFWrVlxxxRWcc845fodrsignCeKQqp4b8HyfqpZLb72fLEEYEzyqyjfffMPs2bMZOXLkmeVNmjShTZs2XH311XTo0IFTp05lsBWTF+TkPgjJ4nNjTB51+vRpvvzyS2bNmsXs2bPZuHEjhQoVomTJktx9990MGzbszI1pVr+oYCiUyfrUpxeZPTfG5CHJycl88sknREdHU716dZo1a8aLL77IRRddxOTJk9m5cyeTJ0/m/fffZ8OGDZw8eZKEhIQz4yub/C3TMwgRqcmfZwqFUj23Mwhj8pgTJ06wdOlSZs2axdy5c9mzZw/FihXjhhtu4NZbb6Vjx46UKfPnMC9Wv6jgyuwaxGmcs4T0EoGqakQoAssquwZhTPqOHTvGwoULmTVrFvPnz+fgwYOULl2ajh07cuutt3LDDTdQsmRJv8M0PsjoGkSGTUyqWkhVI9zHtKawSA7GFGTx8fFERkYSERFBZGQk8fHxABw+fJjp06dz++23U758eW655RY++OADbrnlFubPn8/u3bt5++23ue222yw5mDRZJ2Vj8rDU9yh8+OGH9O3bl3//+9/8+OOPHD9+nEqVKtGzZ086d+7MNddcQ5EiRfwO2+QRliCMycNiY2OZMGECGzduZNSoUSQkJHDq1CkOHDjAfffdR+fOnWnWrBkREXayb7IuR3dShxO7BmEKmt27d1O5cmXKlSvH3r17qVOnDp07d6ZTp078/e9/t3sUjCfZvgZhjAk/iYmJ9O3bl+rVq6OqXHTRRSQkJLB+/XqeeeYZjh07ZvcomKDw3MQkIkWApkBVVZ0hIiUBVPVoqIIzxjhUlaVLl/LCCy+wYMECihUrxl133UWdOnUYP348qkpycrJVQDVB5bXc96XAe8Bx4G/ADOAaoBfwj5BFZ0wBd+LECWbMmMGYMWP45ptvqFixIk899RQDBgygQoUKgDOugt2jYELB65jUK4CJqvqGiOxX1bLuGcQGVQ2LQWHtGoTJT/bv38/EiRMZN24c27dvp169egwZMoQ777zTht00QRWMManrA2+68wpO05KIFA9CfMYY18aNG/nPf/7D1KlTOXr0KG3atGHKlCm0a9furMF1jMkNXhPE70Bj4MxXdBG5Aht21JigWLlyJS+88AJz584lIiKCrl27MmTIEBo0aOB3aKYA85ogHgM+EJFXgKIiMgIYAPQNWWTG5HOnTp1izpw5vPDCC6xatYqyZcsybNgw7r//fqpWrep3eMZ46+aqqu8D7YAKwDLgQuBWVf04pwGIyAMi8r2I/CAig91l5URkkYj87D6Wzel+jAkXhw8f5r///S916tTh9ttvZ/fu3YwbN47NmzczatQoSw4mbHju5qqqXwMDg7lzEYnEOQu5AjgBfCQi7wP9gCWqOlpEhgPDgWHB3LcxoRYfH09sbOyZ3kUDBgxgy5YtTJw4kYMHD9K8eXOef/55OnXqZHc6m7Dk6QxCRGaLyFWpll0lIu/mcP91gS9U9X+qmoxzdnIr0Al4zX3Na8DNOdyPMbkqpUbSuHHjWLVqFVWqVCE6Opp///vftG3bls8//5zPPvuMW2+91ZKDCVteu7nuBSqq6qmAZYWBXap6frZ3LlIXmAc0A44BS3AuhPdQ1TLuawTYn/I81fv74ZxtUL169cabNm3KbijGBFVkZCTPPPMM8+bNIy4ujlKlStG2bVu+//571q9f73d4xpyR7TGpAzawDairqocClpUBflLVyjkMrg9O09VR4Aecm/HuCkwIKfdeZLQduw/ChJOIiAiqVKnCjh07GDp0KCNGjKBkyZIUK1bMaiSZsBKMWkwLgYkicq67wXOBl4CPchqcqsapamNVvRrYD2wAdolIFXdfVYDdOd2PMblh//793HXXXZw+fZoiRYqwatUqnn32WcqUKcOKFSusRpLJU7wmiIeAc4F9IrIb2AecBwzOaQAiUtF9rI5z/eFtnLIevdyX9MJphjImrM2fP5/69evz5ptvcvPNzmWzI0eO2DjOJs/y1ItJVfcDHUSkMlAN2KKqO4MUwywROR84CdynqgdEZDTwjtv8tAm4I0j7Mibo9u3bxwMPPMCbb77JpZdeyvz582ncuDHx8fFWI8nkaVkaD8L9tl8qcJmqbgx2UNlh1yCMH+bOncuAAQPYu3cvjzzyCDExMRQtWtTvsIzxLMe1mESkHRAHVEm1SgHro2cKnD179jBo0CDi4+Np0KABH330EQ0bNvQ7LGOCyus1iPHA00BJVS0UMFlyMAXOu+++S/369Xn33Xd58skn+eqrryw5mHzJ653UZXHKfeeP8UmNyYbdu3dz//33M3PmTBo1asTixYu59NJL/Q7LmJDxegYRB/QOZSDGhCtVZcaMGdSvX5958+YRGxvLqlWrLDmYfM/rGURTYJBbF+ms3kvu/QvG5Es7d+5k4MCBzJkzhyZNmvDqq69Sv359v8MyJld4TRBT3MmYAkFVefvttxk0aBBHjx7l2WefZciQIRQu7Lm+pTF5ntf7IF7L/FXG5A87duxgwIABvPfeezRt2pSpU6faHdCmQPL8dUhEKuGU5S4PnBn7UFWnhiAuY3KdqvLGG2/wwAMPkJSUxPPPP8/gwYOt2qopsLyW+74Z+BV4CpgIRLuPPUIXmjGhEx8fT2RkJBEREURGRjJ+/Hg6duxIr169qF+/PuvWreOhhx6y5GAKNK+9mEYCvVX1cuCo+9gPWBOyyIwJkcCxGo4dO0b79u2Jjo5m0aJFjB07lmXLlnHRRRf5HaYxvvNa7vuQqqZUct2vqmVFpBCwU1UrhjpIL6zUhvEqMjKScePGUadOHe655x4WLlzIpZdeSlJSEhs2bPA7PGNyVY5LbQC7RaSSqu4CfheRZsAerMyGyYMSExM5duwYDRs25NixY4wbN46+fftSokQJv0MzJqx4bWKaDLRw58cCCcA6YEIogjImVE6fPk358uXp0KEDlStXZu3atdx///2sXLnSeioZk4rXbq7PBsy/LiKf4NRlSgxVYMYE2/79++nZsye7d++mZMmSPPfcc9SqVevMWA2xsbF+h2hMWMnWXT+qujnYgRgTSt988w2dO3dm8+bNjBs3jnLlyvHwww/bWA3GZCDdBCEiiapa153fglPa+y9UtXqIYjMmKKZNm8a9995LuXLlWL58Oc2aNQOgW7duPkdmTHjL6Ayib8B891AHYkywJSUl8cADDzBp0iRatmzJ9OnTqVgxLDrdGZMnpJsgVHUFgIhEAHcD/VT1eG4FZkxObNq0idtuu43Vq1czbNgwRo4caXWUjMmiTP9jVPWUiLQFTudCPMbk2Mcff0zXrl1JTk5mzpw53HzzzX6HZEye5LWb61jgSREpEspgjMmJ06dP8/TTT9OuXTuqVq3K6tWrLTkYkwNez7mjgcrAEBH5g4AL1naR2oSD/fv306NHDz744APuvPNOJk6cSMmSJf0Oy5g8zWuCsIvUJmx9/fXXdO7cma1bt/LSSy8xcOBARCTzNxpjMuT1RrlloQ7EmOx49dVXGThwIOeffz7Lly+nadOmfodkTL6RlfEgGgJX8dfxIP4VgriMyVBSUhKDBg1i8uTJtGrVivj4eOvCakyQeR0Poh/wGdAKGAZcCjwE1A5daMakbdOmTbRo0YLJkyczfPhwFi5caMnBmBDwegbxMNBOVT91y33fIiI3AF1CGJsxf7Fw4UK6detGcnIyc+fOpVOnTn6HZEy+5bWba0VV/dSdPy0ihVR1AXBjiOIy5iynT5/mqaee4oYbbuCCCy5gzZo1lhyMCTGvZxBbRaSGqv4ObAA6icge4ETIIjPGtW/fPnr06MGHH35Ijx49eOWVV2zsBmNygdcziOeAlGL5TwFvAkuBJ0MRlCnYAseLrl27NhdffDGLFi1iwoQJvPbaa5YcjMklGZ5BiMg7wDTgdVU9DaCqC0SkLFBUVY/kNAAReRC4B+fmu++A3kAVYDpwPs641z1U1c5WCoCU8aLj4uL4+eefuf/++zl9+jSPP/449957r9/hGVOgZHYGsQ2IA7aLyBgRuQxAVU8EKTlcAAwColQ1EmcI0y7As8BYVa0N7Af65HRfJm+IjY1l4sSJTJ8+nf79+3PNNdcwc+ZMZsyY4XdoxhQ4GSYIVX0QuACnmmtl4HMRWSciQ0SkUpBiKAwUF5HCQAlgB0532nfd9a8BVlCngEhMTOSFF15g0qRJjBgxgo8++oiOHTuSmGiDFxqT2zK9BqGqp1X1Q1XthtP08yLQEdgkIu/nZOequg14HtiMkxgO4jQpHVDVZPdlW3GS1F+ISD8RWS0iq//444+chGLCwN69eznnnHP4+OOPmTRpEqNGjSIiIoIVK1bYeNHG+MDrRWoAVPUQ8CGwANgFXJ2TnbvXMjoBNYGqQEmgXRbimaSqUaoaVaFChZyEYny2efNmWrRowcmTJ6lYsSK1a9fm5MmTZ8aLjomJ8TtEYwocT91cRaQYcCvQC7gW+BR4DJiVw/23AX5T1T/c/cwG/g6UEZHC7lnE33CuhZh86ocffuD666/nyJEjLFmyhG3bthEdHW3jRRvjs8x6MV0L9AQ64zQBvQH0VdXNQdr/ZqCpiJQAjgGtgdVAAnAbTk+mXsC8IO3PhJnPPvuMjh07Urx4cZYvX85ll10GYAnBmDCQWRPTHOA4TpmNS1Q1NojJAVX9Audi9FqcLq6FgEk49Z6GiMgvOF1d44K1TxM+5s+fT5s2bahYsSIrV648kxyMMeEhsyamyqEeh1pVHwceT7V4I3BFKPdr/PXqq6/St29fLr/8cj788EPsGpIx4Sezbq4hTQ6m4FFVRo8ezd13303r1q1JSEiw5GBMmMpSLyZjcuL06dM8+OCDjBgxgm7dujF//nxKlSrld1jGmHSkmyDcLqjGBMWJEyfo3r07L774IoMHD+aNN96gaNGifodljMlARmcQm1JmRGRxLsRi8qnDhw/TsWNH4uPjGT16NGPGjKFQITt5NSbcZXSR+n8iEgkkAleIMwr8X0aCTyniZ0xa/vjjD9q3b8/XX3/N1KlT6d27t98hGWM8yihBPAl8CZzjPk9OtV5wKrBGhCAukw/8/vvvtG3blhcQ66wAAB8zSURBVK1btzJ37lw6duzod0jGmCxIN0Go6ssiMhmnSN9PQH3+TArGZOjbb7+lXbt2JCUlsXjxYpo3b+53SMaYLMrwPgi31MVWEblcVTdl9FpjUixfvpybbrqJ0qVL8+mnn1K/fn2/QzLGZIPXK4W/i8iTIvKbiCSJyEb3uXVDMWeZO3cubdu2pWrVqqxcudKSgzF5WFaGHG0D9AcaAANwxmx4NkRxmTxo0qRJdO7cmcsvv5xPP/2UatWq+R2SMSYHPFVzBW4HGqjqXvf5ehFZC6wDHgxJZCbPUFVGjhzJv/71L9q3b88777xDyZIl/Q7LGJNDXhPEX7q3ZrLcFBCnTp3igQceYPz48fTs2ZMpU6ZQpEgRv8MyxgSB1yammcB8EbleROqKSDtgLvBO6EIz4e748eN07dqV8ePH889//pNp06ZZcjAmH/GaIB4GFgPjcYYEHYczZsOwEMVlwlB8fDyRkZFERERQr149GjVqxMyZM3n++ed57rnncO6lNMbkF54ShKqeUNV/qWptVS2hqnVU9TGr9lpwxMfHExMTw7hx49i0aRMnT57kxx9/ZODAgTz00EN+h2eMCQEriGM8iY2NJS4ujgsvvJBrr72W7du3M3r0aJYtW+Z3aMaYELEEYTxJTEzk/PPP56qrrmL//v0sXbqUIUOGkJiY6HdoxpgQ8dqLyRRwNWvW5Oqrr6ZYsWIsW7aMyMhIEhISqFu3rt+hGWNCxBKEydSqVavYsWMHx48fZ9y4cVx88cUkJCTQp08fYmNj/Q7PGBMinhOEiLQFGgJnDQGmqv8KdlAmfHzyySfceOONVKlShUGDBvHss89y1113UbduXWJjY+natavfIRpjQsRTghCRl4A7cLq2/i9glVV2zcc++ugjbrnlFmrVqsXixYvPJAljTMHg9QyiG06pjS2hDMaEj9mzZ9OlSxciIyP5+OOPKV++vN8hGWNymddeTHuAA6EMxISPN998kzvuuIMmTZqwdOlSSw7GFFBeE8QLwFsi0kxEagVOoQzO5L5JkybRs2dPrrnmGhYuXEiZMmX8DskY4xOvTUwvu4+px4y0IUfzkbFjxzJkyBA6dOjAzJkzKV68uN8hGWN85LXURqF0JksO+YCq8vTTTzNkyBBuv/12Zs+ebcnBGJO1+yBEpDpwAbDVLljnD6rK8OHDee655+jVqxdTpkyhcGG7PcYY4/EMQkSqiMgy4BdgNvCriCwXkaohjc6E1OnTp4mOjua5555j4MCBTJ061ZKDMeYMrxepX8YZPa6sqlYBygJfA6+EKjATWsnJyfTp0+fMWA4vvfQShQpZaS5jzJ+8fl1sAVRR1ZMAqnpURB4GtuVk5yJyMTAjYFEt4F/A6+7yGsDvwB2quj8n+zJ/OnHiBN27d2fmzJk89dRTPProozaWgzHmL7x+ZdwP1Eu17GJyeG+Eqq5X1Yaq2hBojHOX9hxgOLBEVesAS9znJgiSkpLo3LkzM2fO5IUXXuCxxx6z5GCMSZPXM4jngMUiEgdsAi4EegOPBTGW1sCvqrpJRDoB17rLXwM+wUavy7EjR47QqVMnEhISeOWVV+jfv7/fIRljwpinBKGqk0XkV5ySG5cB24FuqrokiLF0AeLd+UqqusOd3wlUSusNItIP6AdQvXr1IIaS/xw4cIAOHTqwatUqXnvtNXr06OF3SMaYMCeq/tfbE5GiOEmnvqruEpEDqlomYP1+VS2b0TaioqJ09erVoQ41T9qzZw/XX3893333HfHx8XTu3NnvkIwxYUJE1qhqVFrr0j2DEJEYVY11559K73VBKvd9A7BWVXe5z3eJSBVV3SEiVYDdQdhHgbRjxw7atGnDxo0bmTdvHjfccIPfIRlj8oiMmpj+FjBfLcRxdOXP5iWA94BewGj3cV6I958vbdq0idatW7Nz504WLFjAtdde63dIxpg8xPcmJhEpCWwGaqnqQXfZ+cA7QHWci+J3qOq+jLZjTUxn+/nnn2ndujWHDx9mwYIFNG3a1O+QjDFhKKMmJq93Uqf54SwiOW76UdWjqnp+SnJwl+1V1daqWkdV22SWHAzEx8cTGRlJREQEderUoUmTJhw7doyEhARLDsaYbPHazbVI6gUiUgSr5BoW4uPjiYmJIS4ujhIlStC2bVuOHj3K6NGjadiwod/hGWPyqAzPIETkUxFZDhRzay+dmYD1wMpcidJkKDY2lri4OM455xzatm1L2bJlmTZtGtOmTfM7NGNMHpbZGcQUQIAmQFzAcgV2AUtDFJfJgsTERGrVqkVUVBSVKlViyZIlVK5cmV69evkdmjEmD8swQajqayISAXQApqvq8dwJy2RF3bp1ueWWW/jf//7H8uXLqVatGgkJCdStW9fv0IwxeVimF6lV9RTQEjgZ+nBMdtSvX5+vv/6agQMHUrt2bRISEujTpw8xMTF+h2aMycO8Fut7HRgQykBM9qxcuZJZs2bRtGlTFixYQLFixYiOjiY2NpauXbv6HZ4xJg/z2ovpCiDaLfG9BecaBACqenUoAjOZO3DgAN26daN69ep89NFHnHfeeX6HZIzJR7wmiMnuZMKEqtKvXz+2bdvGihUrLDkYY4LOazXX10IdiMmaKVOmMHPmTJ555hmuvPJKv8MxxuRDnseYFJHeIrJURNa7j71DGZhJ348//sgDDzxAmzZtePjhh/0OxxiTT3k6gxCRGKAn8AJ/Dhj0sIhUTan4anLHsWPH6NKlC6VKleL111+3caSNMSHj9RrEPcC1qropZYGILASWA5YgctHQoUP57rvvWLBgAVWqVPE7HGNMPub162dJ4I9Uy/YCxYMbjsnInDlzmDBhAg899BDt2rXzOxxjTD7nNUF8BLwlIheLSHERuQRnrOiFoQvNBNq8eTN9+vShcePGjBo1yu9wjDEFgNcEcT9wGPgWOAqscx+jQxSXCZCcnEz37t05efIk06dPp2jRon6HZIwpALx2cz0E9BSRu4DywB5VPR3KwMyfRo4cyaeffsobb7xB7dq1/Q7HGFNAeL1IjYjUAe4AqgLbReQdVf05ZJEZAJYtW8bTTz9Nz5496d69u9/hGGMKEK8jynUDvgYuw2lauhRY6y43IbJ3717uvPNOatWqxUsvveR3OMaYAsbrGcRIoL2qLk9ZICJXAW8Ab4cisIJOVenTpw+7d+/m888/p3Tp0n6HZIwpYLwmiNLA56mWrcLp/mpCYMKECcybN48xY8bQuHFjv8MxxhRAXnsxjQFGiUgxABEpjnOD3JhQBVaQrVu3joceeoj27dszePBgv8MxxhRQoqqZv0hkC1AZp8z3fqAszlCkOwJfp6rVQxCjJ1FRUbp69Wq/dh80R48eJSoqigMHDvDtt99SoUIFv0MyxuRjIrJGVaPSWue1icm6z+SSwYMHs379ehYtWmTJwRjjK6/3QSwLdSAGZsyYwZQpUxgxYgStW7f2OxxjTAHntZtrERF5UkQ2ikiS+/ikiNgtvUHy22+/0a9fP5o2bcqTTz7pdzjGGOO5iek5nGFHB/Bnue/HgHOBB0MTWsFx8uTJM+NHx8fHU6RIEZ8jMsYY7wnidqCBqu51n68XkbU4NZksQeTQ448/zhdffMGMGTOoUaOG3+EYYwzgvZurZHG58WjJkiWMHj2ae+65hzvuuMPvcIwx5gyvCWImMF9ErheRuiLSDpgLvBO60PK/3bt30717dy655BJefPFFv8MxxpizeE0QDwOLgfHAGmAckAAMy2kAIlJGRN4VkZ9EJFFEmolIORFZJCI/u49lc7qfcHP69Gnuuusu9u/fz4wZMyhRooTfIRljzFkyTRAiEgFMBmJVtbaqllDVOqr6mKoeD0IMLwIfqeolQAMgERgOLFHVOsAS93m+8uKLL7JgwQLGjBnDpZde6nc4xhjzF17vpN4BVFfVk0Hduch5wDdALQ0IRETW44yBvUNEqgCfqOrFGW0rL91JvWbNGpo1a0aHDh2YPXs2InYpxxjjj4zupPbaxDQWCMV9DzVxxrp+VUS+FpEpIlISqKSqKWU8dgKV0nqziPQTkdUisvqPP1IPmR2eDh8+TJcuXahUqRJxcXGWHIwxYctrgogG/gkcEpEtIrI5Zcrh/gsDjYCXVfVynLEmzmpOcs8s0jzNUdVJqhqlqlF5pSzFfffdx8aNG3n77bcpV66c3+EYY0y6/K7FtBXYqqpfuM/fxUkQu0SkSkAT0+4Q7T9XxMfHExsby48//oiq0rlzZ6666iq/wzLGmAx5uUjdFmgMFFLVZamnnOxcVXcCW0Qk5fpCa+BH4D2gl7usFzAvJ/vxU3x8PDExMQwbNowSJUpw2WWXsXbtWuLj4/0OzRhjMpThRWoRGQb8C/gBqAcMU9XxQQ1ApCEwBSgKbAR64ySud4DqOKU97lDVfRltJ1wvUkdGRjJ27FhGjBjBxo0bWbduHb/88gvR0dF8//33fodnjCngclLuuz/QWlVXicjfgUk490IEjap+A6QVXL4oZ5qYmMhXX33FmjVrmDVrFtWqVaNy5cokJib6HZoxxmQosyam8qq6CkBVP8MZNMhkQe3atXnyySe5+eabufXWWwFYsWIFdevW9TkyY4zJWKYXqcXph3lmCngOgKqeDl14eZuqUqJECU6ePEmXLl04efIkK1asoE+fPsTGxvodnjHGZCizM4hSQDJwEjgBlAl4nvJo0jFr1iy++eYbunfvztNPP02xYsWIjo4mNjb2THlvY4wJV5mdQdTMlSjyoYMHDzJo0CAuv/xypk6dSuHCXnsUG2NMeMjwU0tVN+VWIPnNo48+ys6dO5k3b54lB2NMnuT1TmqTBV999RXjx4/nvvvuo0mTJn6HY4wx2WIJIsiSk5Pp378/lStXZuTIkX6HY4wx2WZtH0H20ksv8fXXX/POO+9w3nnn+R2OMcZkW5bOIESkmog0DVUwed3WrVt57LHHaN++Pbfddpvf4RhjTI54ShAiUl1EPgN+whlZDhG5TUSmhDK4vGbQoEGcOnWK8ePHWxlvY0ye5/UMYiLwAVCaP+99WARcF4qg8qL33nuPOXPm8Pjjj1OjRg2/wzHGmBzzOqLcXqCCqp4WkX2qWs5dfkBVy4Q6SC/8LNZ35MgR6tWrx3nnncfatWspUqSIL3EYY0xW5aRYX4pdQG1gQ8BG6wE5HTAoX3jiiSfYsmUL06dPt+RgjMk3vDYxPQ+8LyK9gcIi0hWYATwbssjyiHXr1vGf//yHvn370rx5c7/DMcaYoPF0BqGqU91mpv7AFqAn8Jiqzg1lcOHu1KlT9O/fn/PPP5/Ro0f7HY4xxgSVpwQhIhGqOo88PLJbKEyaNIkvvviCN99808aXNsbkO16bmHaKyAR30CAD7Nixg+HDh9OmTRu6devmdzjGGBN0XhNEW+AIEC8iv4nIMyJyaQjjCnsPPvggx48fZ8KECXbPgzEmX/KUIFT1a1V9WFWrA3cBZYGlIvJtKIMLVwsXLmTGjBk88sgj1KlTx+9wjDEmJLJTrO8nIBGni2uNoEaTBxw7doyBAwdy8cUXM2zYML/DMcaYkPF6kboM0BnoBjQFPsbp4vpe6EILTyNHjmTjxo0sXbqUc845x+9wjDEmZLzeKLcdWAm8DXRW1QOhCyl8/fDDD/z73/+mV69etGzZ0u9wjDEmpLwmiP9T1R0hjSTMnT59mgEDBlC6dGmef/55v8MxxpiQSzdBiMjVqrrcfVpXROqm9TpVXRqSyMLMq6++yooVK4iLi6N8+fJ+h2OMMSGXbrE+EfleVSPd+d/Seb+qaq1QBZcVoSzW98cff3DJJZdQv359li1bZt1ajTH5RraK9aUkB3e+ZigCyyuGDh3K4cOHeeWVVyw5GGMKDK8DBqVZYkNEZgc3nPCTkJDA66+/zj//+U/q1avndzjGGJNrvI4HcUhVz01j+ZmxIfwWiiam48ePc9lll5GcnMz3339P8eLFg7p9Y4zxW7bHgxCRp9zZogHzKWoBm4IQ3O/AYeAUkKyqUSJSDqeceA3gd+AOVd2f031l1ejRo9mwYQMLFy605GCMKXAya2Kq5k6FAuarAX/DKft9e5DiaKmqDQOy2HBgiarWAZa4z3PVhg0bGDVqFF26dKFt27a5vXtjjPFdhmcQqtobQERWqurk3AkJgE7Ate78a8AnQK7VtVBV7r33XooXL87YsWNza7fGGBNWvA4YNBlAREoD5QEJWLcxhzEo8LGIKDBRVScBlQJuzNsJVMrhPrLkrbfeYunSpUyYMIHKlSvn5q6NMSZseK3FVBenzEYDnA90cR8BInIYQwtV3SYiFYFFIvJT4EpVVTd5pBVXP6AfQPXq1XMYhmPfvn0MGTKEK6+8kv79+wdlm8YYkxd5reb6MpAAlAMO4ZT7ngj0ymkAqrrNfdwNzAGuAHaJSBUA93F3Ou+dpKpRqhpVoUKFnIYCwPDhw9m3bx8TJ06kUKHsFLs1xpj8wesnYANgmFukT1T1IPBP4Omc7FxESrrNVohISZyBib7HqRKbknx6kUtDna5YsYLJkyfz4IMP0qBBg9zYpTHGhC2vxfqSgCLASWCPiFQH9gPn53D/lYA57t3JhYG3VfUjEfkKeEdE+uB0pb0jh/vJ1IkTJxgwYADVq1fniSeeCPXujDEm7HlNEJ/ifEhPA94FFgDHgRwV6nMvcP/lq7qq7gVa52TbWTVmzBh++OEH3nvvPUqWLJmbuzbGmLDktRdT4Df4R4AfgFLA66EIKrfEx8cTGxtLYmIiqkpUVBQ33nij32EZY0xYyPJVWFU9rapvqOrLqno0FEHlhvj4eGJiYvjvf//LddddxznnnMPu3buJj4/3OzRjjAkLGY0H8QZ/dmVNl6r2DGpEuSQ2Npa4uDj27NnDwoULGTt2LA0aNCA6OpquXbv6HZ4xxvguoyamX3ItCh8kJibSokULli5dSqdOnbj//vtRVRITE/0OzRhjwkJG40E8mZuB5La6deuyYsUKrr/+eq6//nrAKe1dt26aA+cZY0yB4/VO6lbprcurQ47GxMTQp08f4uLiaNGiBStWrKBPnz7Exsb6HZoxxoQFr91c41I9rwAUBbbilP3Oc1KuM0RHR5OYmEjdunWJjY216w/GGOPyNGDQX94kEgE8ChxW1TFBjyobQjkmtTHG5FcZDRiUrWJDqnoKiAUezklgxhhjwldOqtFdB5wOViDGGGPCi9eL1Fs4+56IEkAxYGAogjLGGOM/rxepu6d6fhTYoKqHghyPMcaYMOG1FtOyUAdijDEmvHjqxSQi5wGDgMtxivSdoaptQxNa1ojIHzilwbOjPLAniOH4yY4l/OSX4wA7lnCVk2O5UFXTHHHNaxPTTJyhRecAx7IZREild4BeiMjq9Lp55TV2LOEnvxwH2LGEq1Adi9cE0RQor6ongh2AMcaY8OS1m+sK4JJQBmKMMSa8eD2DuAv4UES+AHYFrlDVp4IdlA8m+R1AENmxhJ/8chxgxxKuQnIsXi9STwZuwhl6NPAahObV8SCMMcZkzGuCOAxcpKo7Qh+SMcaYcOD1GsRG4GQoA/GLiLQTkfUi8ouIDPc7nuwSkWoikiAiP4rIDyLygN8x5YSIRIjI1yLyvt+x5ISIlBGRd0XkJxFJFJFmfseUXSLyoPu39b2IxItIMb9j8kpEporIbhH5PmBZORFZJCI/u49l/YzRi3SO49/u39e3IjJHRMoEa39eE8QbwHsi0lVEWgVOwQrED25V2vHADUA9oKuI1PM3qmxLBh5S1Xo4vc7uy8PHAvAAkB+G93sR+EhVLwEakEePSUQuwLkXKkpVI3G6vXfxN6osmQa0S7VsOLBEVesAS9zn4W4afz2ORUCkql4GbABGBGtnXi9S3+c+jkq1XMmj40G4rgB+UdWNACIyHegE/OhrVNngNv/tcOcPi0gicAF58FhE5G9AB5yKwUN8Difb3BtMr8bp5IHbTTwvdxUvDBQXkZM49di2+xyPZ6q6XERqpFrcCbjWnX8N+AQYlmtBZUNax6GqHwc8XQXcFqz9eS21UTNYOwwzFwBbAp5vBa70KZagcf+ALge+8DeSbPsPTin50n4HkkM1gT+AV0WkAbAGeEBVj/obVtap6jYReR7YjNNR5eNUH0x5UaWA66o7gUp+BhMkdwMzgrWxnJT7NmFIREoBs4DBebGYooh0BHar6hq/YwmCwkAj4GVVvRynyGVeaMb4C7d9vhNO0qsKlBSR1EU88yx1eutkffS0MCIiMThNzW8Fa5ueEoSIbBGRzWlNwQrEJ9uAagHP/+Yuy5NEpAhOcnhLVWf7HU82/R24SUR+B6YDrUTkTX9DyratwFZVTTmTexcnYeRFbYDfVPUPVT0JzAaa+xxTTu0SkSoA7uNun+PJNhG5C+gI3KnZGSY0Hdkt910F5yLi9GAF4pOvgDoiUhMnMXQBuvkbUvaIiOCMHZ4YLsPAZoeqjsC9yCYi1wJDVTVPflNV1Z3ul6uLVXU90Jo8eE3ItRloKiIlcJqYWgN5fYzf94BewGj3cZ6/4WSPiLTDaZK9RlX/F9RtZzfZiEhlnN4ZDYMZUG4TkfY4bd4RwFRVjfU5pGwRkRY4NzJ+x58j/T2iqh/6F1XOBCSIjn7Hkl0i0hCYAhTF6S7eW1X3+xtV9ojIk8A/cJoxvgbuUdXj/kbljYjE41yQLo9TDeJxYC7wDlAdpxL0Haq6z68YvUjnOEYA5wB73ZetUtUBQdlfDhJEWeB3VT0vGIEYY4wJL16HHE1db6kE0B5YEPSIjDHGhAWv1yCqpXp+FBiDcwOdMcaYfCjbTUzGGGPytwy7uYrI30Xk2XTWjRaRpqEJyxhjjN8yuw/iEWB5Ous+AWKCGo0xxpiwkVmCaAh8lM66xUDj4IZjzNlE5IiI5Kjel4g8IiJTghSPikjtYGzLmHCXWYI4F6f/dlqKkPdr5ZhcJiK/i8gx94N/l4hMc8uDpElVS6UUU8wuVR2lqvfkZBteicj1IrJcRA6LyB8iskxEbsqNfYcD9/fbxu84THBkliB+Atqms66tu96YrLpRVUvhlJ2IAh5N/QIR8drDLmyIyG3ATOB1nLItlYB/ATf6GZcx2ZVZghgLTBSRW0WkEICIFBKRW4FXcLq6GpMtqroN516aSDjTfHOfiPwM/BywrLY7P01ExovIB+439C9E5P9Stici9d2BX/a5ZyePuMufSKnnJCI13G32E5HtIrJDRIYGbOMKEflcRA64614SkfTOogl4n+D8PzytqlNU9aCqnlbVZara131NIRF5VEQ2iTPoy+tuSfDAuHq75Tn2i8gAEWkizkAwB0TkpYD93SUin7nxHRRnwJjWAeurish77s/iFxHpG7DuCRF5x93/YXEGAYpK9d5Z7hnQbyIyyMt7ReQNnLuS57tniA97+DMw4UxVM5xw6vEfxqljv919PAwMyey9NtmUegJ+B9q489WAH3A+VMGpprkIKAcUD1hW252fhlNO4Aqce3jeAqa760rjjIfxEFDMfX6lu+4J4E13voa7zXigJHApTknulJga4wy4VNh9bSJOZVxSx5PquC5x19XM4NjvBn7BGUOlFE7BuzdSxfWKG39bIAmnHERFnNL0u3Hq7YAzxkQy8CBOc+8/gINAOXf9cmCCu62G7jG2Cvh5JOHc7BoBPINTngGcL41rcM58irqxbgSuz+y9qX+/NuX9yduLnGsR1+MUsrseONfvwG3Km5P7AXIEOIBT/2ZCqmTQKtXrUyeIKQHr2gM/ufNdga/T2WdaCeKSgPXPAXHpvHcwMCeteFK97u/uumIZHPsSYGDA84txhvItHBDXBQHr9wL/CHieUsY9JUFsx72XyV32JdADJ/GeAkoHrHsGmBbw81gcsK4ecMydvxLYnCruEcCrmb034PdrCSKfTF4HDDoELPTyWmM8uFlVF6ezbks6y1PsDJj/H843cXA+FH/NQgyB+9mEcyaBiFyE01QUhVNSpjDON+rMpBRKqwL8ls5rqrr7CtxvYc4eqGZXwPyxNJ4HXtDfpu6ncsD2qrrTPlU9nGpdVMDz1D/HYu51nwuBqiJyIGB9BE4hyAzfq6rJmHzFBgwy4Sa7t/ZvIWvD3waWj6nOn8NnvozT+aKOqp6Lcy+QeNjeejeGzhm8ZjvOB3DgfpM5OwlkxQXutY/A7W13p3IiUjrVOi9jnWzBGfehTMBUWlXbe4zJSjPkI5YgTH7xPlBFRAaLyDkiUlpEMho+9jERKSEi9YHe/DlMY2ngEHBERC4B7vWyc/eb/BB3u71F5Fz3onQLEZnkviweeFBEarpde0cBM3LwzbsiMEhEiojI7UBd4ENV3QKsBJ4RkWIichnQB/Ay8NKXwGERGSYixUUkQkQiRaSJx5h2kbfHqTcBLEGYfMFtTrkOp0vpTpxeUC0zeMsynAvGS4Dn9c/xlYfiXGs7DEwmC+P7quq7OBeL78b5Fr8LGMmfA9FMxSlwuRynGSoJiPa6/TR8AdQB9gCxwG2qmtLU1RXnusZ2YA7weAbNeoHHcApnZLKGbox7cMaz8FrW/xngUbfX1dBMX23CmhXrMwWKiNTA+eArkpfbzMUZYvIeVW3hdywm/7IzCGOMMWmyBGGMMSZN1sRkjDEmTXYGYYwxJk2WIIwxxqTJEoQxxpg0WYIwxhiTJksQxhhj0mQJwhhjTJr+H6ZgCyMS4rOEAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xrnhkl02lxBq",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "9452a60e-268e-422f-b42b-f5c6f8ce147c"
      },
      "source": [
        "# Renaming Columns\n",
        "\n",
        "# We will rename the columns of the loading matrix that was generated for each observation using PCA. \n",
        "# After renaming, we will select 7 principal components \n",
        "# and make a data frame with the dependent variable and the 7 PCs.\n",
        "\n",
        "pca_train = pd.DataFrame(pca_train,columns=['PC_' + str(i) for i in range(1, 14)])\n",
        "pca_train.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PC_1</th>\n",
              "      <th>PC_2</th>\n",
              "      <th>PC_3</th>\n",
              "      <th>PC_4</th>\n",
              "      <th>PC_5</th>\n",
              "      <th>PC_6</th>\n",
              "      <th>PC_7</th>\n",
              "      <th>PC_8</th>\n",
              "      <th>PC_9</th>\n",
              "      <th>PC_10</th>\n",
              "      <th>PC_11</th>\n",
              "      <th>PC_12</th>\n",
              "      <th>PC_13</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-2.206607</td>\n",
              "      <td>-1.550518</td>\n",
              "      <td>1.694059</td>\n",
              "      <td>1.829224</td>\n",
              "      <td>-2.345380</td>\n",
              "      <td>-0.699030</td>\n",
              "      <td>1.130661</td>\n",
              "      <td>-0.469812</td>\n",
              "      <td>0.581400</td>\n",
              "      <td>-0.291183</td>\n",
              "      <td>0.419261</td>\n",
              "      <td>0.129598</td>\n",
              "      <td>0.005218</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-2.895220</td>\n",
              "      <td>0.632332</td>\n",
              "      <td>-0.117836</td>\n",
              "      <td>0.029440</td>\n",
              "      <td>0.111146</td>\n",
              "      <td>0.039596</td>\n",
              "      <td>0.070625</td>\n",
              "      <td>0.811305</td>\n",
              "      <td>0.094068</td>\n",
              "      <td>-0.318388</td>\n",
              "      <td>-0.168244</td>\n",
              "      <td>0.272926</td>\n",
              "      <td>0.198293</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-1.328565</td>\n",
              "      <td>-0.869346</td>\n",
              "      <td>-0.677164</td>\n",
              "      <td>-0.369238</td>\n",
              "      <td>0.301381</td>\n",
              "      <td>0.432975</td>\n",
              "      <td>0.884145</td>\n",
              "      <td>0.255710</td>\n",
              "      <td>-0.509150</td>\n",
              "      <td>0.629288</td>\n",
              "      <td>0.456284</td>\n",
              "      <td>-0.268324</td>\n",
              "      <td>0.137592</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2.894334</td>\n",
              "      <td>0.058086</td>\n",
              "      <td>0.311875</td>\n",
              "      <td>-0.535148</td>\n",
              "      <td>-0.709886</td>\n",
              "      <td>-0.038247</td>\n",
              "      <td>-1.237339</td>\n",
              "      <td>-0.022792</td>\n",
              "      <td>-0.471815</td>\n",
              "      <td>-0.210664</td>\n",
              "      <td>0.106260</td>\n",
              "      <td>0.289861</td>\n",
              "      <td>0.100084</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3.388655</td>\n",
              "      <td>0.337146</td>\n",
              "      <td>0.722763</td>\n",
              "      <td>-0.542509</td>\n",
              "      <td>-0.758414</td>\n",
              "      <td>0.939876</td>\n",
              "      <td>-0.556901</td>\n",
              "      <td>-0.403837</td>\n",
              "      <td>-0.204815</td>\n",
              "      <td>-0.488596</td>\n",
              "      <td>0.338563</td>\n",
              "      <td>-0.027698</td>\n",
              "      <td>0.055221</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       PC_1      PC_2      PC_3  ...     PC_11     PC_12     PC_13\n",
              "0 -2.206607 -1.550518  1.694059  ...  0.419261  0.129598  0.005218\n",
              "1 -2.895220  0.632332 -0.117836  ... -0.168244  0.272926  0.198293\n",
              "2 -1.328565 -0.869346 -0.677164  ...  0.456284 -0.268324  0.137592\n",
              "3  2.894334  0.058086  0.311875  ...  0.106260  0.289861  0.100084\n",
              "4  3.388655  0.337146  0.722763  ...  0.338563 -0.027698  0.055221\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bYZqPwfNmA1Z",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "91e2376e-ca63-4467-c57b-b5d8464b4a54"
      },
      "source": [
        "# Concatenate Dependent variable and Principal Components\n",
        "\n",
        "# We now concatenate the dependent variable i.e. \"MEDV\" with principal components and \n",
        "# Suppose take the first seven components for our analysis. \n",
        "# First, we will reset the index for Y_train as we need to concatenate datasets to make one whole train dataset. \n",
        "# Then we will remove the index variable from the dataset \n",
        "# and make a subset of the dataset having 7 PCs and the dependent variable.\n",
        "\n",
        "Y_train1 = Y_train.reset_index()\n",
        " \n",
        "pca_train1 = pd.concat([pca_train,Y_train1],axis=1)\n",
        "pca_train2 = pca_train1.drop(columns='index')\n",
        "pca_train3 = pca_train1[['PC_1', 'PC_2', 'PC_3', 'PC_4', 'PC_5', 'PC_6', 'PC_7','MEDV']]\n",
        "pca_train3.head()\n",
        "\n",
        "# After this step, run any alogirthim as per your choice !!"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PC_1</th>\n",
              "      <th>PC_2</th>\n",
              "      <th>PC_3</th>\n",
              "      <th>PC_4</th>\n",
              "      <th>PC_5</th>\n",
              "      <th>PC_6</th>\n",
              "      <th>PC_7</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-2.206607</td>\n",
              "      <td>-1.550518</td>\n",
              "      <td>1.694059</td>\n",
              "      <td>1.829224</td>\n",
              "      <td>-2.345380</td>\n",
              "      <td>-0.699030</td>\n",
              "      <td>1.130661</td>\n",
              "      <td>35.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-2.895220</td>\n",
              "      <td>0.632332</td>\n",
              "      <td>-0.117836</td>\n",
              "      <td>0.029440</td>\n",
              "      <td>0.111146</td>\n",
              "      <td>0.039596</td>\n",
              "      <td>0.070625</td>\n",
              "      <td>25.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-1.328565</td>\n",
              "      <td>-0.869346</td>\n",
              "      <td>-0.677164</td>\n",
              "      <td>-0.369238</td>\n",
              "      <td>0.301381</td>\n",
              "      <td>0.432975</td>\n",
              "      <td>0.884145</td>\n",
              "      <td>36.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2.894334</td>\n",
              "      <td>0.058086</td>\n",
              "      <td>0.311875</td>\n",
              "      <td>-0.535148</td>\n",
              "      <td>-0.709886</td>\n",
              "      <td>-0.038247</td>\n",
              "      <td>-1.237339</td>\n",
              "      <td>16.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3.388655</td>\n",
              "      <td>0.337146</td>\n",
              "      <td>0.722763</td>\n",
              "      <td>-0.542509</td>\n",
              "      <td>-0.758414</td>\n",
              "      <td>0.939876</td>\n",
              "      <td>-0.556901</td>\n",
              "      <td>10.9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       PC_1      PC_2      PC_3      PC_4      PC_5      PC_6      PC_7  MEDV\n",
              "0 -2.206607 -1.550518  1.694059  1.829224 -2.345380 -0.699030  1.130661  35.2\n",
              "1 -2.895220  0.632332 -0.117836  0.029440  0.111146  0.039596  0.070625  25.0\n",
              "2 -1.328565 -0.869346 -0.677164 -0.369238  0.301381  0.432975  0.884145  36.2\n",
              "3  2.894334  0.058086  0.311875 -0.535148 -0.709886 -0.038247 -1.237339  16.1\n",
              "4  3.388655  0.337146  0.722763 -0.542509 -0.758414  0.939876 -0.556901  10.9"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    }
  ]
}