self_in_Python_Demystified.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<small><small><i>\n",
    "All the IPython Notebooks in **Python Object Class** lecture series by **[Dr. Milaan Parmar](https://www.linkedin.com/in/milaanparmar/)** are available @ **[GitHub](https://github.com/milaan9/06_Python_Object_Class)**\n",
    "</i></small></small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# self in Python, Demystified\n",
    "\n",
    "If you have been programming in Python (object-oriented programming) for some time, then you have definitely come across methods that have **`self`** as their first parameter.\n",
    "\n",
    "Let us first try to understand what this recurring **`self`** parameter is."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is `self` in Python?\n",
    "\n",
    "In object-oriented programming, whenever we define methods for a class, we use **`self`** as the first parameter in each case. Let's look at the definition of a class called **`Cat`**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:00.891349Z",
     "start_time": "2021-08-08T07:37:00.879143Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "class Cat:\n",
    "    def __init__(self, name, age):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "\n",
    "    def info(self):\n",
    "        print(f\"I am a cat. My name is {self.name}. I am {self.age} years old.\")\n",
    "\n",
    "    def make_sound(self):\n",
    "        print(\"Meow\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Explanation:**\n",
    "\n",
    "In this case all the methods, including **`__init__`**, have the first parameter as **`self`**.\n",
    "\n",
    "We know that class is a blueprint for the objects. This blueprint can be used to create multiple numbers of objects. Let's create two different objects from the above class.\n",
    "\n",
    "```python\n",
    "cat1 = Cat('Amelia', 3)\n",
    "cat2 = Cat('Bella', 6)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **`self`** keyword is used to represent an instance (object) of the given class. In this case, the two **`Cat`** objects **`cat1`** and **`cat2`** have their own **`name`** and **`age`** attributes. If there was no **`self`** argument, the same class couldn't hold the information for both these objects.\n",
    "\n",
    "However, since the class is just a blueprint, **`self`** allows access to the attributes and methods of each object in python. This allows each object to have its own attributes and methods. Thus, even long before creating these objects, we reference the objects as **`self`** while defining the class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why is self explicitly defined everytime?\n",
    "\n",
    "Even when we understand the use of **`self`**, it may still seem odd, especially to programmers coming from other languages, that **`self`** is passed as a parameter explicitly every single time we define a method. As **The Zen of Python** goes, **\"Explicit is better than implicit\"**.\n",
    "\n",
    "So, why do we need to do this? Let's take a simple example to begin with. We have a **`Point`** class which defines a method **`distance`** to calculate the distance from the origin."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.030017Z",
     "start_time": "2021-08-08T07:37:00.899161Z"
    }
   },
   "outputs": [],
   "source": [
    "class Point(object):\n",
    "    def __init__(self,x = 0,y = 0):\n",
    "        self.x = x\n",
    "        self.y = y\n",
    "\n",
    "    def distance(self):\n",
    "        \"\"\"Find distance from origin\"\"\"\n",
    "        return (self.x**2 + self.y**2) ** 0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now instantiate this class and find the distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.215566Z",
     "start_time": "2021-08-08T07:37:01.033439Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10.816653826391969"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p1 = Point(6,9)\n",
    "p1.distance()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example, **`__init__()`** defines three parameters but we just passed two (6 and 9). Similarly **`distance()`** requires one but zero arguments were passed. Why is Python not complaining about this argument number mismatch?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What Happens Internally?\n",
    "\n",
    "**`Point.distance`** and **`p1.distance`** in the above example are different and not exactly the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.434806Z",
     "start_time": "2021-08-08T07:37:01.239982Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "function"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(Point.distance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.554923Z",
     "start_time": "2021-08-08T07:37:01.444571Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "method"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(p1.distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the first one is a function and the second one is a method. A peculiar thing about methods (in Python) is that the object itself is passed as the first argument to the corresponding function.\n",
    "\n",
    "In the case of the above example, the method call **`p1.distance()`** is actually equivalent to **`Point.distance(p1)`**.\n",
    "\n",
    "Generally, when we call a method with some arguments, the corresponding class function is called by placing the method's object before the first argument. So, anything like **`obj.meth(args)`** becomes **`Class.meth(obj, args)`**. The calling process is automatic while the receiving process is not (its explicit).\n",
    "\n",
    "This is the reason the first parameter of a function in class must be the object itself. Writing this parameter as **`self`** is merely a convention. It is not a keyword and has no special meaning in Python. We could use other names (like **`this`**) but it is highly discouraged. Using names other than **`self`** is frowned upon by most developers and degrades the readability of the code (**Readability counts**)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Self Can Be Avoided\n",
    "\n",
    "By now you are clear that the object (instance) itself is passed along as the first argument, automatically. This implicit behavior can be avoided while making a **static** method. Consider the following simple example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.693592Z",
     "start_time": "2021-08-08T07:37:01.560784Z"
    }
   },
   "outputs": [],
   "source": [
    "class A(object):\n",
    "\n",
    "    @staticmethod\n",
    "    def stat_meth():\n",
    "        print(\"Look no self was passed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, **`@staticmethod`** is a **[function decorator](https://github.com/milaan9/07_Python_Advanced_Topics/blob/main/004_Python_Decorators.ipynb)** that makes **`stat_meth()`** static. Let us instantiate this class and call the method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.804923Z",
     "start_time": "2021-08-08T07:37:01.697500Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Look no self was passed\n"
     ]
    }
   ],
   "source": [
    "a = A()\n",
    "a.stat_meth()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above example, we can see that the implicit behavior of passing the object as the first argument was avoided while using a static method. All in all, static methods behave like the plain old functions (Since all the objects of a class share static methods)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:01.895743Z",
     "start_time": "2021-08-08T07:37:01.808830Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "function"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(A.stat_meth)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.016833Z",
     "start_time": "2021-08-08T07:37:01.906487Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "function"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(a.stat_meth)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Self Is Here To Stay\n",
    "\n",
    "The explicit **`self`** is not unique to Python. This idea was borrowed from **Modula-3**. Following is a use case where it becomes helpful.\n",
    "\n",
    "There is no explicit variable declaration in Python. They spring into action on the first assignment. The use of **`self`** makes it easier to distinguish between instance attributes (and methods) from local variables.\n",
    "\n",
    "In the first example, **`self.x`** is an instance attribute whereas **`x`** is a local variable. They are not the same and they lie in different namespaces.\n",
    "\n",
    "Many have proposed to make **`self`** a keyword in Python, like **`this`** in C++ and Java. This would eliminate the redundant use of explicit **`self`** from the formal parameter list in methods.\n",
    "\n",
    "While this idea seems promising, it is not going to happen. At least not in the near future. The main reason is backward compatibility. Here is a blog from the creator of Python himself explaining **[why the explicit self has to stay](http://neopythonic.blogspot.in/2008/10/why-explicit-self-has-to-stay.html)**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `__init__()` is not a constructor\n",
    "\n",
    "One important conclusion that can be drawn from the information so far is that the **`__init__()`** method is not a constructor. Many naive Python programmers get confused with it since **`__init__()`** gets called when we create an object.\n",
    "\n",
    "A closer inspection will reveal that the first parameter in **`__init__()`** is the object itself (object already exists). The function **`__init__()`** is called immediately **after** the object is created and is used to initialize it.\n",
    "\n",
    "Technically speaking, a constructor is a method which creates the object itself. In Python, this method is **`__new__()`**. A common signature of this method is:\n",
    "\n",
    "```python\n",
    "__new__(cls, *args, **kwargs)\n",
    "```\n",
    "\n",
    "When **`__new__()`** is called, the class itself is passed as the first argument automatically(cls).\n",
    "\n",
    "Again, like **`self`**, **`cls`** is just a naming convention. Furthermore, __*args__ and __**kwargs__ are used to take an arbitrary number of arguments during method calls in Python.\n",
    "\n",
    "Some important things to remember when implementing **`__new__()`** are:\n",
    "\n",
    "* **`__new__()`** is always called before **`__init__()`**.\n",
    "* First argument is the class itself which is passed implicitly.\n",
    "* Always return a valid object from **`__new__()`**. Not mandatory, but its main use is to create and return an object.\n",
    "\n",
    "Let's take a look at an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.095450Z",
     "start_time": "2021-08-08T07:37:02.021231Z"
    }
   },
   "outputs": [],
   "source": [
    "class Point(object):\n",
    "\n",
    "    def __new__(cls,*args,**kwargs):\n",
    "        print(\"From new\")\n",
    "        print(cls)\n",
    "        print(args)\n",
    "        print(kwargs)\n",
    "\n",
    "        # create our object and return it\n",
    "        obj = super().__new__(cls)\n",
    "        return obj\n",
    "\n",
    "    def __init__(self,a = 0,b = 0):\n",
    "        print(\"From init\")\n",
    "        self.a = a\n",
    "        self.b = b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's now instantiate it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.187737Z",
     "start_time": "2021-08-08T07:37:02.099357Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From new\n",
      "<class '__main__.Point'>\n",
      "(6, 9)\n",
      "{}\n",
      "From init\n"
     ]
    }
   ],
   "source": [
    "p2 = Point(6,9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example illustrates that **`__new__()`** is called before **`__init__()`**. We can also see that the parameter **`cls`** in **`__new__()`** is the class itself (**`Point`**). Finally, the object is created by calling the **`__new__()`** method on **object** base class.\n",
    "\n",
    "In Python, object is the base class from which all other classes are derived. In the above example, we have done this using **[super()](https://github.com/milaan9/04_Python_Functions/blob/main/002_Python_Functions_Built_in/068_Python_super%28%29.ipynb)**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use `__new__` or `__init__`?\n",
    "\n",
    "You might have seen **`__init__()`** very often but the use of **`__new__()`** is rare. This is because most of the time you don't need to override it. Generally, **`__init__()`** is used to initialize a newly created object while **`__new__()`** is used to control the way an object is created.\n",
    "\n",
    "We can also use **`__new__()`** to initialize attributes of an object, but logically it should be inside **`__init__()`**.\n",
    "\n",
    "One practical use of **`__new__()`**, however, could be to restrict the number of objects created from a class.\n",
    "\n",
    "Suppose we wanted a class **`HexPoint`** for creating instances to represent the six vertices of a square. We can inherit from our previous class **`Point`** (the second example in this article) and use **`__new__()`** to implement this restriction. Here is an example to restrict a class to have only four instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.279042Z",
     "start_time": "2021-08-08T07:37:02.192619Z"
    }
   },
   "outputs": [],
   "source": [
    "class HexPoint(Point):\n",
    "    MAX_Inst = 6\n",
    "    Inst_created = 0\n",
    "\n",
    "    def __new__(cls,*args,**kwargs):\n",
    "        if (cls.Inst_created >= cls.MAX_Inst):\n",
    "            raise ValueError(\"Cannot create more objects\")\n",
    "        cls.Inst_created += 1\n",
    "        return super().__new__(cls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.371816Z",
     "start_time": "2021-08-08T07:37:02.284903Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n",
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n",
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n",
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n",
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n",
      "From new\n",
      "<class '__main__.HexPoint'>\n",
      "()\n",
      "{}\n",
      "From init\n"
     ]
    }
   ],
   "source": [
    "p1 = HexPoint(0,0)\n",
    "p2 = HexPoint(1,0)\n",
    "p3 = HexPoint(1,1)\n",
    "p4 = HexPoint(0,1)\n",
    "p5 = HexPoint(2,2)\n",
    "p6 = HexPoint(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-08-08T07:37:02.912350Z",
     "start_time": "2021-08-08T07:37:02.378165Z"
    }
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Cannot create more objects",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-14-fab2e1af2c65>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mp7\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mHexPoint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m2\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m4\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m<ipython-input-12-e15e5601926f>\u001b[0m in \u001b[0;36m__new__\u001b[1;34m(cls, *args, **kwargs)\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__new__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcls\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mcls\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mInst_created\u001b[0m \u001b[1;33m>=\u001b[0m \u001b[0mcls\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMAX_Inst\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m             \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Cannot create more objects\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      8\u001b[0m         \u001b[0mcls\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mInst_created\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__new__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcls\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Cannot create more objects"
     ]
    }
   ],
   "source": [
    "p7 = HexPoint(2,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}