chapter-10-organizing-files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dbdc6aa6",
   "metadata": {},
   "source": [
    "# Chapter 10: ORGANIZING FILES"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fa9dfaa",
   "metadata": {},
   "source": [
    "## The `shutil` Module"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a00ed4a9",
   "metadata": {},
   "source": [
    "### Copying Files and Folders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b7d454a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7f15d3b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[01;32mREADME.md\u001b[0m*                                     \u001b[01;32mdata.txt\u001b[0m*\r\n",
      "\u001b[01;32mchapter-06-manipulating-strings.ipynb\u001b[0m*         \u001b[34;42mfiles\u001b[0m/\r\n",
      "\u001b[01;32mchapter-09-paths-reading-writing-files.ipynb\u001b[0m*  \u001b[34;42mmy_folder\u001b[0m/\r\n",
      "\u001b[01;32mchapter-10-organizing-files.ipynb\u001b[0m*\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914c7d03",
   "metadata": {},
   "source": [
    "`shutil.copy(`**`file, destination`**`)`\n",
    "\n",
    "copy a single file to the folder at the path *destination*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2616c879",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'my_folder/data.txt'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.copy(\"data.txt\", \"my_folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0862bf1c",
   "metadata": {},
   "source": [
    "`shutil.copytree(`**`files, destination`**`)`\n",
    "\n",
    "copy the folder at the path *source*, along with all of its files and subfolders, to the folder at the path *destination*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8061b43f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'backup_folder'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.copytree(\"files\", \"backup_folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ebbcce1",
   "metadata": {},
   "source": [
    "### Moving and Renaming Files and Folders"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d55b88d9",
   "metadata": {},
   "source": [
    "`shutil.move(`**`source, destination`**`)`\n",
    "\n",
    "Recursively move a file or directory to another location. This is similar to the Unix \"mv\" command. Return the file or directory's destination."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6382c6e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'new_data.txt'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.move(\"data.txt\", \"new_data.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1e2c81e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'files/data.txt'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.move(\"data.txt\", \"files\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "4948ebd2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'new_folder'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shutil.move(\"files\", \"new_folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe7c4c4e",
   "metadata": {},
   "source": [
    "### Permanently Deleting Files and Folders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "74c54c9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.remove(\"temp_data.txt\")  # Remove a file (same as unlink())."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "a5f6415b",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.rmdir(\"this_empty_folder\")  # Remove a directory. Directory must be empty."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "70704ad1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Recursively delete a directory tree.\n",
    "shutil.rmtree(\"my_folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89780329",
   "metadata": {},
   "source": [
    "### Safe Deletes with the `send2trash` Module"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eee73d21",
   "metadata": {},
   "source": [
    "**`send2trash`** sends *folders and files* to computer’s trash or recycle bin instead of permanently deleting them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "9ed9555c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: send2trash in /home/javohir/anaconda3/lib/python3.9/site-packages (1.8.0)\r\n"
     ]
    }
   ],
   "source": [
    "!pip install send2trash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ad87d551",
   "metadata": {},
   "outputs": [],
   "source": [
    "from send2trash import send2trash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8676fe71",
   "metadata": {},
   "outputs": [],
   "source": [
    "send2trash(\"my_data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27be76f4",
   "metadata": {},
   "source": [
    "**Note** that the `send2trash()` function can only send files to the recycle bin; it cannot pull files out of it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "994b0032",
   "metadata": {},
   "source": [
    "## Walking a Directory Tree"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df06c25",
   "metadata": {},
   "source": [
    "**`os.walk`** - Directory tree generator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35131a4f",
   "metadata": {},
   "source": [
    "##### This figure shows an example **Calibre Library** folder with its contents:\n",
    "\n",
    "**Calibre Library**\\\n",
    "| &emsp; metadata.db\\\n",
    "| &emsp; metadata_db_prefs_backup.json\\\n",
    "|\\\n",
    "\\\\---**John Schember**\\\n",
    "&emsp; \\\\---**Quick Start Guide**\\\n",
    "&emsp;&emsp;&emsp; cover.jpg\\\n",
    "&emsp;&emsp;&emsp; metadata.opf\\\n",
    "&emsp;&emsp;&emsp; Quick Start Guide - John Schember.epub\n",
    "\n",
    "---\n",
    "the figure generated with `tree /a /f` command in Windows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "79594bd3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current folder: /mnt/d/Calibre Library/\n",
      "SUBFOLDER OF /mnt/d/Calibre Library/: John Schember\n",
      "FILE INSIDE /mnt/d/Calibre Library/: metadata.db\n",
      "FILE INSIDE /mnt/d/Calibre Library/: metadata_db_prefs_backup.json\n",
      "\n",
      "Current folder: /mnt/d/Calibre Library/John Schember\n",
      "SUBFOLDER OF /mnt/d/Calibre Library/John Schember: Quick Start Guide\n",
      "\n",
      "Current folder: /mnt/d/Calibre Library/John Schember/Quick Start Guide\n",
      "FILE INSIDE /mnt/d/Calibre Library/John Schember/Quick Start Guide: cover.jpg\n",
      "FILE INSIDE /mnt/d/Calibre Library/John Schember/Quick Start Guide: metadata.opf\n",
      "FILE INSIDE /mnt/d/Calibre Library/John Schember/Quick Start Guide: Quick Start Guide - John Schember.epub\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for folderName, subfolders, filenames in os.walk('/mnt/d/Calibre Library/'):\n",
    "    print(f\"Current folder: {folderName}\")\n",
    "    \n",
    "    for subfolder in subfolders:\n",
    "        print(f\"SUBFOLDER OF {folderName}: {subfolder}\")\n",
    "        \n",
    "    for filename in filenames:\n",
    "        print(f\"FILE INSIDE {folderName}: {filename}\")\n",
    "    \n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01d60f94",
   "metadata": {},
   "source": [
    "## Compressing Files with the `zipfile` Module"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4961debd",
   "metadata": {},
   "source": [
    "### Reading ZIP Files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "22e3e753",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['spam.txt', 'cats/', 'cats/catnames.txt', 'cats/zophie.jpg']"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import zipfile\n",
    "\n",
    "# reading a zip file\n",
    "exampleZip = zipfile.ZipFile(\"automate-online-materials/example.zip\", 'r')\n",
    "exampleZip.namelist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "d3d0f102",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<ZipInfo filename='spam.txt' compress_type=deflate external_attr=0x2020 file_size=13908 compress_size=3828>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spamInfo = exampleZip.getinfo(\"spam.txt\")\n",
    "spamInfo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1df986de",
   "metadata": {},
   "source": [
    "sizes in *bytes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "a9e0a57b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13908"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spamInfo.file_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "70af9bf9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3828"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spamInfo.compress_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "efefa9b7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Compressed size is 3.63x smaller!'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f\"Compressed size is {round(spamInfo.file_size / spamInfo.compress_size, 2)}x smaller!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "a9575619",
   "metadata": {},
   "outputs": [],
   "source": [
    "exampleZip.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c7ddc42",
   "metadata": {},
   "source": [
    "### Extracting from ZIP Files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "0bb3fd45",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['spam.txt', 'cats/', 'cats/catnames.txt', 'cats/zophie.jpg']"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import zipfile\n",
    "\n",
    "# reading a zip file\n",
    "exampleZip = zipfile.ZipFile(\"automate-online-materials/example.zip\", 'r')\n",
    "exampleZip.namelist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "9c18055f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/mnt/d/spam_folder/spam.txt'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extract a member from the archive to the current working directory.\n",
    "exampleZip.extract(\"spam.txt\", \"/mnt/d/spam_folder\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "a07596b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract all members from the archive to the current working directory.\n",
    "exampleZip.extractall(\"/mnt/d/my_files\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "3e081e68",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Close the file\n",
    "exampleZip.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f642da62",
   "metadata": {},
   "source": [
    "### Creating and Adding to ZIP Files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "81f5d0ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "import zipfile\n",
    "\n",
    "# open a zip file in write mode\n",
    "newZip = zipfile.ZipFile(\"new.zip\", 'w')\n",
    "newZip.write(\"data.txt\", compress_type=zipfile.ZIP_DEFLATED)\n",
    "newZip.write(\"data.txt\")  # works faster but it does not compress\n",
    "newZip.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f379c761",
   "metadata": {},
   "source": [
    "***!!! Keep in mind*** that, just as with writing to files, `write mode` will erase all existing contents of a ZIP file. If you want to simply add files to an existing ZIP file, pass `'a'` as the second argument to `zipfile.ZipFile()` to open the ZIP file in `append mode`.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af02fbff",
   "metadata": {},
   "source": [
    "## Project: Renaming Files with American-Style Dates to European-Style Dates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "id": "921c8866",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import re\n",
    "import shutil\n",
    "\n",
    "datePattern = re.compile(r\"\"\"^(.*?)     # all text before the date\n",
    "                        ((0|1)?\\d)-     # one or two digits for the month\n",
    "                        ((0|1|2|3)?\\d)- # one or two digits for the day\n",
    "                        ((19|20)\\d\\d)   # four digits for the year\n",
    "                        (.*?)$          # all text after the date\n",
    "                        \"\"\", re.VERBOSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "377146dc",
   "metadata": {},
   "source": [
    "`re.VERBOSE` for the second argument will allow whitespace and comments in the regex string to make it more readable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "id": "f6054ccd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Renaming \"0-15-2014.py\" to \"15-0-2014.py\"...\n",
      "Renaming \"1-1-1999-buggyAddingProgram.py\" to \"1-1-1999-buggyAddingProgram.py\"...\n",
      "Renaming \"2-29-2013.zip\" to \"29-2-2013.zip\"...\n",
      "Renaming \"automate-01-09-2001-requirements.txt\" to \"automate-09-01-2001-requirements.txt\"...\n",
      "Renaming \"bir12-31-2022thdays.py\" to \"bir31-12-2022thdays.py\"...\n",
      "Renaming \"mouseNow_5-20-1976.py\" to \"mouseNow_20-5-1976.py\"...\n",
      "Renaming \"spam4-4-1984.txt\" to \"spam4-4-1984.txt\"...\n"
     ]
    }
   ],
   "source": [
    "# Loop over the files in the working directory\n",
    "for amer_file in os.listdir():\n",
    "    match = re.search(datePattern, amer_file)\n",
    "\n",
    "    # Skip files without a date.\n",
    "    if match == None:\n",
    "        continue\n",
    "\n",
    "    # Get the different parts of the filename\n",
    "    before_date = match.group(1)\n",
    "    month       = match.group(2)\n",
    "    day         = match.group(4)\n",
    "    year        = match.group(6)\n",
    "    after_date  = match.group(8)\n",
    "\n",
    "    # Form the European-style filename\n",
    "    euro_file = f'{before_date}{day}-{month}-{year}{after_date}'\n",
    "    print(f'Renaming \"{amer_file}\" to \"{euro_file}\"...')\n",
    "\n",
    "    # Get the full, absolute file paths\n",
    "    current_path = os.getcwd()\n",
    "    amer_file = os.path.join(current_path, amer_file)\n",
    "    euro_file = os.path.join(current_path, euro_file)\n",
    "\n",
    "    # Rename the files\n",
    "    shutil.move(amer_file, euro_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32ae08b8",
   "metadata": {},
   "source": [
    "### Ideas for Similar Programs\n",
    "\n",
    "- To add a prefix to the start of the filename, such as adding *spam_* to\n",
    "rename *eggs.txt* to *spam_eggs.txt*\n",
    "- To change filenames with European-style dates to American-style\n",
    "dates\n",
    "- To remove the zeros from files such as *spam0042.txt*\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb59d9f0",
   "metadata": {},
   "source": [
    "## Project: Backing Up a Folder into a ZIP File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "2cdabb49",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/mnt/d/GitHub/automate-the-boring-stuff/My_Project\r\n"
     ]
    }
   ],
   "source": [
    "!pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "1d6c5dbd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 1\t\t   'New folder1.zip'   dir1\t\t    spam01.txt\r\n",
      " My_Project_1.zip  'New folder2.zip'   folder1\t\t    temp\r\n",
      "'New folder'\t   'New folder3.zip'   hello.app\r\n",
      "'New folder1'\t   'New folder4.zip'   messages879352.TXT\r\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "f65b0677",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New ZIP file's name: My_Project_2.zip\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import re\n",
    "import zipfile\n",
    "\n",
    "# STEP 1: Figure Out the ZIP File’s Name\n",
    "folder = os.path.basename(os.getcwd())\n",
    "zip_num = 1\n",
    "\n",
    "while True:\n",
    "    new_zip_name = f'{folder}_{zip_num}.zip'\n",
    "    if not os.path.exists(new_zip_name):\n",
    "        break\n",
    "    zip_num += 1\n",
    "\n",
    "print(\"New ZIP file's name:\", new_zip_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "acd74bac",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adding ./hello.app...\n",
      "Adding ./messages879352.TXT...\n",
      "Adding ./spam01.txt...\n",
      "Adding ./1/2/3/LAST!/important.txt...\n",
      "Adding ./folder1/app.txt...\n",
      "Adding ./folder1/hello.app...\n",
      "Adding ./New folder/spam012 - Copy.txt...\n",
      "Adding ./New folder/spam012.txt...\n",
      "Adding ./New folder/dir1 - Copy/spam012 - Copy (2).txt...\n",
      "Adding ./New folder/dir1 - Copy/spam012 - Copy (3).txt...\n",
      "Adding ./New folder/dir1 - Copy (2)/spam012 - Copy (2).txt...\n",
      "Adding ./New folder/dir1 - Copy (2)/spam012 - Copy - Copy.txt...\n",
      "Adding ./New folder1/spam012 - Copy.txt...\n",
      "Adding ./New folder1/spam012.txt...\n",
      "Adding ./New folder1/dir1 - Copy/spam012 - Copy (2).txt...\n",
      "Adding ./New folder1/dir1 - Copy/spam012 - Copy (3).txt...\n",
      "Adding ./New folder1/dir1 - Copy (2)/spam012 - Copy (2).txt...\n",
      "Adding ./New folder1/dir1 - Copy (2)/spam012 - Copy - Copy.txt...\n",
      "Done.\n"
     ]
    }
   ],
   "source": [
    "# STEP 2: Create the New ZIP File\n",
    "new_zip = zipfile.ZipFile(new_zip_name, mode='w')\n",
    "\n",
    "# STEP 3: Walk the Directory Tree and Add to the ZIP File\n",
    "# Walk the entire folder tree and compress the files in each folder.\n",
    "for folder, subfolders, files in os.walk(\".\"):\n",
    "    for file in files:\n",
    "        if not file.endswith('.zip'):  # don't back up the backup ZIP files\n",
    "            print(f'Adding {folder}/{file}...')\n",
    "            new_zip.write(os.path.join(folder, file), compress_type=zipfile.ZIP_DEFLATED)\n",
    "new_zip.close()\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abbe7470",
   "metadata": {},
   "source": [
    "### Ideas for Similar Programs\n",
    "\n",
    "- Walk a directory tree and archive just files with certain extensions, such as *.txt* or *.py*, and nothing else.\n",
    "- Walk a directory tree and archive every file except the *.txt* and *.py* ones.\n",
    "- Find the folder in a directory tree that has the greatest number of files or the folder that uses the most disk space.\n",
    "\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c32e7417",
   "metadata": {},
   "source": [
    "## Practice Projects"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79e4bdf0",
   "metadata": {},
   "source": [
    "### Selective Copy\n",
    "\n",
    "Write a program that walks through a folder tree and searches for files with a certain file extension (such as *.pdf* or *.jpg*). Copy these files from whatever location they are in to a new folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4e043c06",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import os\n",
    "import shutil"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6496ac8c",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying: messages879352.TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: important.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: app.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (10).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (2).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (3).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (4).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (5).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (6).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (7).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (8).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy (9).TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352 - Copy.TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: messages879352.TXT -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (10).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (2).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (3).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (4).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (5).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (6).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (7).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (8).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy (9).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01 - Copy.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam01.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (2).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (3).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (2).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy - Copy.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (2).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (3).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy (2).txt -> /mnt/d/GitHub/FOLDER/\n",
      "Copying: spam012 - Copy - Copy.txt -> /mnt/d/GitHub/FOLDER/\n",
      "Done.\n"
     ]
    }
   ],
   "source": [
    "path_to_copy = \"/mnt/d/GitHub/FOLDER/\"\n",
    "\n",
    "# organize the list of required file extensions\n",
    "extensions_to_search = ['txt', 'pdf']\n",
    "# create regex pattern for catching file extension\n",
    "file_name = re.compile(r\"^.*\\.(.+)$\")\n",
    "\n",
    "# iterate over files inside working directory\n",
    "for folder, subfolders, files in os.walk(\".\"):\n",
    "    for file in files:\n",
    "        match = file_name.search(file)\n",
    "\n",
    "        if match:\n",
    "            # get file extension as lower case\n",
    "            file_ext = match.group(1).lower()\n",
    "\n",
    "            # copy file if it has the extension we are looking for\n",
    "            if file_ext in extensions_to_search:\n",
    "                print(f\"Copying: {file} -> {path_to_copy}\")\n",
    "                file_src = os.path.join(folder, file)\n",
    "                shutil.copy(os.path.abspath(file_src), path_to_copy)\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6e593c7",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Deleting Unneeded Files\n",
    "\n",
    "Write a program that walks through a folder tree and searches for\n",
    "exceptionally large files or folders — say, ones that have a file size of more\n",
    "than 100MB. (Remember that to get a file’s size, you can use\n",
    "`os.path.getsize()` from the `os` module.) Print these files with their absolute\n",
    "path to the screen.\n",
    "\n",
    "Find the folder in a directory tree that has the greatest number of files or the folder that uses the most disk space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f07e3fed",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "54a47756",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# organizing list for folders in current directory\n",
    "folders = [item for item in os.listdir() if os.path.isdir(item)]\n",
    "\n",
    "file_num_dict = {}\n",
    "file_size_dict = {}\n",
    "\n",
    "for folder in folders:\n",
    "    # hold the number of counting files\n",
    "    num = 0\n",
    "    # hold the size of files\n",
    "    size = 0\n",
    "\n",
    "    for directory, _, files in os.walk(folder):\n",
    "        # iterate files inside current directory\n",
    "        for file in files:\n",
    "            # count file\n",
    "            num += 1\n",
    "            # get full path\n",
    "            file_path = os.path.join(directory, file)\n",
    "            # calculate folder size by adding every file size\n",
    "            size += os.path.getsize(file_path)\n",
    "\n",
    "        # add folder name and number files to file numbers dictionary\n",
    "        file_num_dict[os.path.basename(folder)] = num\n",
    "        # add folder name and size in MB to file sizes dictionary\n",
    "        file_size_dict[folder] = round(size/1024/1024, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "46892ba7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'1': 1,\n",
       " 'dir1': 0,\n",
       " 'folder1': 2,\n",
       " 'important_files': 132,\n",
       " 'New folder': 6,\n",
       " 'New folder1': 6,\n",
       " 'temp': 0}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_num_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "8b8f0941",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'1': 0.0,\n",
       " 'dir1': 0.0,\n",
       " 'folder1': 0.0,\n",
       " 'important_files': 0.161,\n",
       " 'New folder': 0.0,\n",
       " 'New folder1': 0.0,\n",
       " 'temp': 0.0}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_size_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "77253076",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The greatest number of files folder: \"important_files\", 132 files.\n",
      "The largest size folder: \"important_files\", size: 0.161 MB.\n"
     ]
    }
   ],
   "source": [
    "# sorting dictionaries by values\n",
    "sorted_files = sorted(file_num_dict.items(),\n",
    "                        key=lambda item: item[1],\n",
    "                        reverse=True)\n",
    "\n",
    "sorted_sizes = sorted(file_size_dict.items(),\n",
    "                        key=lambda item: item[1],\n",
    "                        reverse=True)\n",
    "# printing results\n",
    "print(f\"The greatest number of files folder: \\\"{sorted_files[0][0]}\\\", {sorted_files[0][1]} files.\")\n",
    "print(f\"The largest size folder: \\\"{sorted_sizes[0][0]}\\\", size: {sorted_sizes[0][1]} MB.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "191f3b18",
   "metadata": {},
   "source": [
    "> **Note!** The program calculates number of files, sizes of FOLDERS ONLY IN CURRENT DIRECTORY, not nested folders.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a7bf5b",
   "metadata": {},
   "source": [
    "### Filling in the Gaps\n",
    "\n",
    "Write a program that finds all files with a given prefix, such as spam001.txt, spam002.txt, and so on, in a single folder and locates any gaps in the numbering (such as if there is a spam001.txt and spam003.txt but no spam002.txt). Have the program rename all the later files to close this gap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b14f398",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "2c7666c1320a956fa05893d10993164efa76d3af81fc7470d6f4943526f3df36"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}