Opening-and-Reading-Files-Folders.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "\n",
    "<a href='https://www.udemy.com/user/joseportilla/'><img src='../Pierian_Data_Logo.png'/></a>\n",
    "___\n",
    "<center><em>Content Copyright by Pierian Data</em></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Opening and Reading Files\n",
    "\n",
    "So far we've discussed how to open files manually, one by one. Let's explore how we can open files programatically. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "_____"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Review: Understanding File Paths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Marcial\\\\Pierian-Data-Courses\\\\Complete-Python-3-Bootcamp\\\\12-Advanced Python Modules'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Practice File\n",
    "\n",
    "We will begin by creating a practice text file that we will be using for demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "f = open('practice.txt','w+')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "f.write('test')\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting Directories\n",
    "\n",
    "Python has a built-in [os module](https://docs.python.org/3/library/os.html) that allows us to use operating system dependent functionality.\n",
    "\n",
    "You can get the current directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Marcial\\\\Pierian-Data-Courses\\\\Complete-Python-3-Bootcamp\\\\12-Advanced Python Modules'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Listing Files in a Directory\n",
    "\n",
    "You can also use the os module to list directories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt',\n",
       " 'practice.txt']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# In your current directory\n",
    "os.listdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['admin.DESKTOP-O64BPTC',\n",
       " 'All Users',\n",
       " 'Default',\n",
       " 'Default User',\n",
       " 'defaultuser0',\n",
       " 'desktop.ini',\n",
       " 'Marcial',\n",
       " 'Public']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# In any directory you pass\n",
    "os.listdir(\"C:\\\\Users\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Moving Files \n",
    "\n",
    "You can use the built-in **shutil** module to to move files to different locations. Keep in mind, there are permission restrictions, for example if you are logged in a User A, you won't be able to make changes to the top level Users folder without the proper permissions, [more info](https://stackoverflow.com/questions/23253439/shutil-movescr-dst-gets-me-ioerror-errno-13-permission-denied-and-3-more-e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import shutil"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Marcial\\\\practice.txt'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.move('practice.txt','C:\\\\Users\\\\Marcial')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Marcial\\\\Pierian-Data-Courses\\\\Complete-Python-3-Bootcamp\\\\12-Advanced Python Modules\\\\practice.txt'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.move('C:\\\\Users\\\\Marcial\\practice.txt',os.getcwd())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt',\n",
       " 'practice.txt']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deleting Files\n",
    "____\n",
    "**NOTE: The os module provides 3 methods for deleting files:**\n",
    "* os.unlink(path) which deletes a file at the path your provide\n",
    "* os.rmdir(path) which deletes a folder (folder must be empty) at the path your provide\n",
    "* shutil.rmtree(path) this is the most dangerous, as it will remove all files and folders contained in the path.\n",
    "**All of these methods can not be reversed! Which means if you make a mistake you won't be able to recover the file. Instead we will use the send2trash module. A safer alternative that sends deleted files to the trash bin instead of permanent removal.**\n",
    "___\n",
    "\n",
    "Install the send2trash module with:\n",
    "\n",
    "    pip install send2trash\n",
    "    \n",
    "at your command line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import send2trash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt',\n",
       " 'practice.txt']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "send2trash.send2trash('practice.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Walking through a directory\n",
    "\n",
    "Often you will just need to \"walk\" through a directory, that is visit every file or folder and check to see if a file is in the directory, and then perhaps do something with that file. Usually recursively walking through every file and folder in a directory would be quite tricky to program, but luckily the os module has a direct method call for this called os.walk(). Let's explore how it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Marcial\\\\Pierian-Data-Courses\\\\Complete-Python-3-Bootcamp\\\\12-Advanced Python Modules'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['.ipynb_checkpoints',\n",
       " '00-Collections-Module.ipynb',\n",
       " '01-Datetime-Module.ipynb',\n",
       " '01-Opening-and-Reading-Files.ipynb',\n",
       " '02-Math-and-Random-Module.ipynb',\n",
       " '03-Python Debugger (pdb).ipynb',\n",
       " '04-Timing your code - timeit.ipynb',\n",
       " '05-Overview-of-Regular-Expressions.ipynb',\n",
       " '06-Unzipping-and-Zipping-Files.ipynb',\n",
       " '07-OS-Module.ipynb',\n",
       " '08-Advanced-Python-Module-Exercise',\n",
       " 'comp_file.zip',\n",
       " 'Example_Top_Level',\n",
       " 'extracted_content',\n",
       " 'new_file.txt',\n",
       " 'new_file2.txt']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currently looking at folder: Example_Top_Level\n",
      "\n",
      "\n",
      "THE SUBFOLDERS ARE: \n",
      "\t Subfolder: Mid-Example-One\n",
      "\t Subfolder: Mid-Example-Two\n",
      "\n",
      "\n",
      "THE FILES ARE: \n",
      "\t File: Mid-Example.txt\n",
      "\n",
      "\n",
      "Currently looking at folder: Example_Top_Level\\Mid-Example-One\n",
      "\n",
      "\n",
      "THE SUBFOLDERS ARE: \n",
      "\t Subfolder: Bottom-Level-One\n",
      "\t Subfolder: Bottom-Level-Two\n",
      "\n",
      "\n",
      "THE FILES ARE: \n",
      "\t File: Mid-Level-Doc.txt\n",
      "\n",
      "\n",
      "Currently looking at folder: Example_Top_Level\\Mid-Example-One\\Bottom-Level-One\n",
      "\n",
      "\n",
      "THE SUBFOLDERS ARE: \n",
      "\n",
      "\n",
      "THE FILES ARE: \n",
      "\t File: One_Text.txt\n",
      "\n",
      "\n",
      "Currently looking at folder: Example_Top_Level\\Mid-Example-One\\Bottom-Level-Two\n",
      "\n",
      "\n",
      "THE SUBFOLDERS ARE: \n",
      "\n",
      "\n",
      "THE FILES ARE: \n",
      "\t File: Bottom-Text-Two.txt\n",
      "\n",
      "\n",
      "Currently looking at folder: Example_Top_Level\\Mid-Example-Two\n",
      "\n",
      "\n",
      "THE SUBFOLDERS ARE: \n",
      "\n",
      "\n",
      "THE FILES ARE: \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for folder , sub_folders , files in os.walk(\"Example_Top_Level\"):\n",
    "    \n",
    "    print(\"Currently looking at folder: \"+ folder)\n",
    "    print('\\n')\n",
    "    print(\"THE SUBFOLDERS ARE: \")\n",
    "    for sub_fold in sub_folders:\n",
    "        print(\"\\t Subfolder: \"+sub_fold )\n",
    "    \n",
    "    print('\\n')\n",
    "    \n",
    "    print(\"THE FILES ARE: \")\n",
    "    for f in files:\n",
    "        print(\"\\t File: \"+f)\n",
    "    print('\\n')\n",
    "    \n",
    "    # Now look at subfolders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "___\n",
    "Excellent, you should now be aware of how to work with a computer's files and folders in whichever directory they are in. Remember that the os module works for any oeprating system that supports Python, which means these commands will work across Linux,MacOs, or Windows without need for adjustment."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}