NumPy/Lesson 2 - Dimensions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dimensions & Shapes\n",
    "\n",
    "First things first. Let us understand what a matrix really is within Python. Is it a list, is it a tuple? Let's find out.\n",
    "\n",
    "To start, lets start by creating a list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5]\n"
     ]
    }
   ],
   "source": [
    "list1 = [1,2,3,4,5]\n",
    "print(list1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a list, let us convert this to an ndarray."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4 5]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "mat1 = np.array(list1)\n",
    "print(mat1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What changed? Nothing actually. `mat1` is exactly same as `list1`. Well, not really. A lot has changed. They both appear the same when you print them, but they are actually completely different objects. \n",
    "\n",
    "`list1` is of type `list`, where as `mat1` is of type `ndarray`. The `ndarray` is a class within the NumPy package. \n",
    "\n",
    "Let us confirm this once by looking at the specification of the two variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat1?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, so what we just created as a list, is actually a 1D array in NumPy. Let us understand more about 1D arrays in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1D Array \n",
    "\n",
    "So, how do we define a one dimenstional array, or 1D array? \n",
    "\n",
    "*An array is said to be in 1D(one-dimension) if either the number of columns or rows are 1.* \n",
    "\n",
    "Yes, our `mat1` is a 1D array, because it has one row and 5 columns. \n",
    "\n",
    "In linear algebra, a 1D array is simply called a **vector**. We might interchangeably use the term `vector` and `1D array`, so get used to it. Both mean the same thing. \n",
    "\n",
    "Now, what if we wanted to created a 1D array that has 1 column and 5 rows. How would we do that? And we are going to do that using a `list` first and then using `ndarray`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1]\n",
      " [2]\n",
      " [3]\n",
      " [4]]\n"
     ]
    }
   ],
   "source": [
    "list2 = [[1],[2],[3],[4]]\n",
    "mat2 = np.array(list2)\n",
    "print(mat2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interesting uh? What just happened? In a literal sense, the line `list2 = [[1],[2],[3],[4]]` actually created 5 independent lists. \n",
    "\n",
    "1. [1] -> first list\n",
    "2. [2] -> second list\n",
    "3. [3] -> third list\n",
    "4. [4] -> fourth list\n",
    "5. [....] -> fifth list (the outer main list)\n",
    "\n",
    "\n",
    "To create a vertical 1D array, or as you can say a column vector, we basically added lists within the primary list. We created the main list, but instead of adding plain numbers `1`, `2`, `3`, `4`, to the list, we put the numbers in a list of their own, each of size 1, and then added each of the lists `[1]`, `[2]`, `[3]` and `[4]` to the primary list to create a 1D column array. \n",
    "\n",
    "Let us spend some more time to understand the datatype of each item here. We have to very clearly understand what is actually going on here. To slice a matrix, find a transpose, interting matrices, the fundamental concepts of how the data is actually arranged in a matrix how to be very clear. \n",
    "\n",
    "Let us look at the type of `list2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "list2?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, `list2` is of type `list`.\n",
    "\n",
    "Now what about the type of the element within the list? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "list2item1 = list2[0]\n",
    "list2item1?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`list2item1` which represents the first element of `list2` is also of type `list`. Just to be sure, how does this compare to the `list1item1`? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1item1 = list1[0]\n",
    "list1item1?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And there we go. The `list1item1` is actually of type `int` and not `list`. This clearly explains that to create a row array we created a single list, but to create a column array, we created 5 lists. Some developers coming from C/C++ might call a vertical 1D matrix, as actually a 2D matrix. You would not be entirely incorrect in doing so.\n",
    "\n",
    "Let us see how to create a 2D matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2D Array \n",
    "\n",
    "An array that has **1D arrays as its elements** is called a 2D array. This is same as the column array that we saw above, but the inner arrays typically have more than one element each. Below is an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 1]\n",
      " [2 2]\n",
      " [3 3]\n",
      " [4 4]]\n"
     ]
    }
   ],
   "source": [
    "list2d = [[1,1],[2,2],[3,3],[4,4]]\n",
    "\n",
    "mat2d = np.array(list2d)\n",
    "\n",
    "print(mat2d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first created a 2 dimensional list, and then converted that list into an `ndarray`. As a result, what we have effectively got is an array that has 4 rows, and each row has 2 columns. \n",
    "\n",
    "Keep in mind, each row is actually an independent list. \n",
    "\n",
    "``` Python\n",
    "list2d = [[1,1],[2,2],[3,3],[4]]\n",
    "mat2d = np.array(list2d)\n",
    "```\n",
    "\n",
    "Do you think we could use an uneven combination like the one shown above? The last list has just 1 element in it instead of 2. The answer is, you cannot do this. We must past the right combination for `ndarray` to be able to matrix from it; else Python will throw an error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why use 2D arrays?\n",
    "\n",
    "It is simple actually. Most of the datasets you use will be in 2D form. If you have data in Excel and you want to read the Excel data, the same will be in 2D. You can easily represent any tabular data (row & column) in 2D, and most of the structured data you encounter will be in 2D form. Excel files and CSV files, are the most commonly encountered forms of 2D data. \n",
    "\n",
    "Here are some example datasets:<br>\n",
    "1) [Titanic dataset](https://www.kaggle.com/c/titanic)<br>\n",
    "2) [Auto-MPG dataset](https://www.kaggle.com/uciml/autompg-dataset)<br>\n",
    "3) [Credit Card Fraud Detection dataset](https://www.kaggle.com/mlg-ulb/creditcardfraud)<br>\n",
    "\n",
    "We can represent all of these datasets as 2D matrices. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3D Array \n",
    "\n",
    "An array that has **2D arrays as its elements** is called a 3D array. As you can see in the below code block, There are two 2D arrays stacked up on top of each other as elements. The process to create a 3D array is same as that of creating a 2D array. Let's look at an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[1 1]\n",
      "  [2 2]\n",
      "  [3 3]\n",
      "  [4 4]]\n",
      "\n",
      " [[5 5]\n",
      "  [6 6]\n",
      "  [7 7]\n",
      "  [8 8]]]\n"
     ]
    }
   ],
   "source": [
    "list2d_1 = [[1,1],[2,2],[3,3],[4,4]]\n",
    "list2d_2 = [[5,5],[6,6],[7,7],[8,8]]\n",
    "\n",
    "list3d = [list2d_1, list2d_2]\n",
    "\n",
    "mat3d = np.array(list3d)\n",
    "print(mat3d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 1  2  3]\n",
      "  [ 4  5  6]]\n",
      "\n",
      " [[ 7  8  9]\n",
      "  [10 11 12]]]\n"
     ]
    }
   ],
   "source": [
    "arr3d = np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])\n",
    "print(arr3d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have basically combined 2 different 2D arrays to make a 3D array. Again, all lists must be appropriately sized to make a full formed 3D matrix; else we will receive an error.\n",
    "\n",
    "## Why use 3D arrays?\n",
    "\n",
    "Well there is lot of content that often comes in 3D form. We basically have rows and columns of data, but then each cell in this 2D matrix has more than one property associated with it; and that makes up the 3rd dimension. \n",
    "\n",
    "An example use case of a 3D array is image data. An image consists of a 2 dimensional space of pixels, and then each pixel has an RGB value. The RGB stands for Red, Green and Blue colors. Combinations of the RGB colors make the effective color of the pixel. \n",
    "\n",
    "Let's take a look at an image, and how the data corresponding to it would appear.\n",
    "\n",
    "An **image** is simply a matrix of pixels. (Checkout the below image)\n",
    "![3D](./images/Numpy_lsn2.jpg)\n",
    "\n",
    "Each element in the above matrix is a pixel value in that image.\n",
    "\n",
    "<br> In the shape, <br>1) 500 - Width of the image.<br>2) 800 - Height of the image.<br>3) 3 - Color channel of the image.<br>\n",
    "\n",
    "For each image, there's a channel(RGB - Red, Green, Blue) associated to it which will give information of colour to the image. Hence a combination of a pixel co-ordinate and the channel produces an image with vivid colours. Therefore, images are 3 dimensional matrices.\n",
    "\n",
    "## How RGB color codes work\n",
    "\n",
    "The combination of 3 colors can supposedly create any color that we see. Our computer displays, televisions sets, render all colors as a combination of RGB values. The value of each color goes in numercial form from 0 to 255. \n",
    "\n",
    "So R = 0 would mean the color R is fully absent, and R = 255 would mean the color R is fully present. \n",
    "\n",
    "(R = 255, G = 255, B = 255) would mean that the colors Red, Green and Blue are mixed in equal proportions of 33% each to make the effective color. The effective color produced by mixing all in equal proportions is White.\n",
    "\n",
    "(R = 0, G = 0, B = 0) would mean no color is mixed. If all colors are absent, then we actually do not have a color. Or effectively, we do have a Black color. \n",
    "\n",
    "(R = 127, G = 127, B = 127) what would this mean? This means all colors are mixed equally. So would we get White? The answer is Yes and No. Since all are mixed equally the color has to be white. But the intensity of each color in computer form represents brightness. This means we have a white, but that is only half as bring as the (255,255,255). What does less bright white make? The answer is grey. We would thereby have a 50% grey shade. (255,255,255) would similarly be 100% grey, but 100% grey is called white. \n",
    "\n",
    "Let's take a few more examples: \n",
    "\n",
    "> (R = 255, G = 0, B = 0) would make solid Red color\n",
    "\n",
    "> (R = 0, G = 255, B = 0) would make solid Green color\n",
    "\n",
    "> (R = 0, G = 0, B = 255) would make solid Blue color\n",
    "\n",
    "> (R = 127, G = 0, B = 0) would make dark Red color\n",
    "\n",
    "> (R = 50, G = 0, B = 0) would make an even darker shade of Red\n",
    "\n",
    "\n",
    "## Converting to Greyscale\n",
    "\n",
    "In many ML projects, you will convert RGB images to greyscale images. Greyscale are nothing but black and white images. Such shade only make use of the color grey. 0% grey would mean black, and 100% grey would mean white. Between these would exist several other shades of grey, with a total of 256 grey shades possible.\n",
    "\n",
    "Since to produce the color grey, we will always use equal proportions of Red, Green and Blue; we can represent greyscale images by a single number.\n",
    "\n",
    "50% grey, which is (127,127,127) could actually be represented by just the number 127. And this is the standard convention. For grey scale images we specify a single value for the % of grey used, in the range of 0 to 255, without specifying the individual RGB values. \n",
    "\n",
    "\n",
    "## Grey scale images are 2D\n",
    "\n",
    "Since we don't need RGB values for greyscale images; we can fit the pixel data of a greyscale image in a 2D matrix. The rows and columns represent pixes, and the each pixel will then hold an integer value in the range of 0 to 255.\n",
    "\n",
    "```\n",
    "[[0,0],[0,0]]\n",
    "```\n",
    "\n",
    "The above list can represent an image of size 2x2. It is a greyscale image. There are 4 pixels in the image, and all pixes are in black color. So what we have here is a 2x2 square sized black image. \n",
    "\n",
    "\n",
    "```\n",
    "[[255,255], [255,255]]\n",
    "```\n",
    "\n",
    "The above list represents a similar 2x2 greyscale image, but this one is a square white image. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**From above,** \n",
    "\n",
    "1) In 1 dimensional matrix(mat_1), **(6,)** denotes that the matrix is one-dimensional and there are 6 rows in it.<br>\n",
    "2) In 3 dimensional matrix(mat_3), **(2,2,3)** denotes that there are 2 matrices of shape (2,3). \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# More than 3 dimensions\n",
    "\n",
    "As the level of complexity increases with the data, the dimensions keep increasing and algebraically we call it **N-Dimensions**. We can use `ndarray` to make arrays of any number of dimensions. \n",
    "\n",
    "To make a 4 dimensional array, we would simply put 3D arrays into a list; and so on and so forth to make any number of dimensions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Determining Shape & Dimensions\n",
    "\n",
    "The number of dimensions and items in an array is defined by its shape, which is a tuple of N non-negative integers that specify the sizes of each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each n-dimensional array(ndarray).\n",
    "\n",
    "Let's take a look."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4, 2)\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "list2d = [[1,1],[2,2],[3,3],[4,4]]\n",
    "mat2d = np.array(list2d)\n",
    "\n",
    "print(mat2d.shape)\n",
    "print(mat2d.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shape is returned in a form of a Tuple. We use the `shape` function to fetch the shape of a matrix. Here 4 is the number of rows, and 2 is the number of columns. We are basically looking at a 4x2 matrix. \n",
    "\n",
    "We also use the `ndim` function to determine the number of dimensions of the matrix. Here it returns `2` which means this is a 2D matrix.\n",
    "\n",
    "Let us try something interesting. Getting the shape of a 1D matrix. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4]\n",
      "(4,)\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "mat1 = np.array([1,2,3,4])\n",
    "print(mat1)\n",
    "print(mat1.shape)\n",
    "print(mat1.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we got is very interesting. We would have assumed for this matrix to have 1 row and 4 columns. This is not actually true. The shape of the matrix is actually 4 rows and no second dimension, which is represented as `(4,)`. This also effectively is same as a matrix with 1 row and 4 columns.\n",
    "\n",
    "We can confirm that this is a 1D matrix, as the `ndim` function returns the value `1`\n",
    "\n",
    "Let's try a vertical matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1]\n",
      " [2]\n",
      " [3]\n",
      " [4]]\n",
      "(4, 1)\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "mat2 = np.array([[1],[2],[3],[4]])\n",
    "print(mat2)\n",
    "print(mat2.shape)\n",
    "print(mat2.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we see that the shape function officially considers this a 2D matrix. The shape is `(4,1)`, which means 4 rows and 1 column. The `(4,)` and `(4,1)` mean very different types of data. We need to understand this fully and keep this in mind at all times. The `ndim` function also returns the value `2`, thereby indeed confirming that this is a 2D matrix.\n",
    "\n",
    "Let's now try to get the shape of a 3D matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[1]\n",
      "  [2]]\n",
      "\n",
      " [[1]\n",
      "  [2]]]\n",
      "(2, 2, 1)\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "mat3 = np.array([[[1],[2]],[[1],[2]]])\n",
    "print(mat3)\n",
    "print(mat3.shape)\n",
    "print(mat3.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above example clearly denotes, that this is a 3D matrix; where in there are 2 individual matrices each of shape `(2,1)`. \n",
    "\n",
    "So out of curiosity, how would we create a `(2,2,2)` matrix?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[1 1]\n",
      "  [2 2]]\n",
      "\n",
      " [[1 1]\n",
      "  [2 2]]]\n",
      "(2, 2, 2)\n"
     ]
    }
   ],
   "source": [
    "mat3 = np.array([[[1,1],[2,2]],[[1,1],[2,2]]])\n",
    "print(mat3)\n",
    "print(mat3.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There you go. We have a (2,2,2) matrix. This is also the equivalent of a Cube. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}