Automate the Boring Stuff with Python, 2nd Edition (Al Sweigart) (z-library.sk, 1lib.sk, z-lib.sk)

AUTOMATE THE BORING STUFF WITH PYTHON 2ND EDITION Practical Programming for Total Beginners by Al Sweigart San Francisco

AUTOMATE THE BORING STUFF WITH PYTHON, 2ND EDITION. Copyright © 2020 by Al Sweigart. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-10: 1-59327-992-2 ISBN-13: 978-1-59327-992-9 Publisher: William Pollock Production Editor: Laurel Chun Cover Illustration: Josh Ellingson Interior Design: Octopod Studios Developmental Editors: Frances Saux and Jan Cash Technical Reviewers: Ari Lacenski and Philip James Copyeditors: Kim Wimpsett, Britt Bogan, and Paula L. Fleming Compositors: Susan Glinert Stevens and Danielle Foster Proofreaders: Lisa Devoto Farrell and Emelie Burnette Indexer: BIM Indexing and Proofreading Services For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 245 8th Street, San Francisco, CA 94103 phone: 1.415.863.9900; info@nostarch.com www.nostarch.com The Library of Congress Control Number for the first edition is: 2014953114 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/us/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

For my nephew Jack

About the Author Al Sweigart is a software developer and tech book author. Python is his favorite programming language, and he is the developer of several open source modules for it. His other books are freely available under a Creative Commons license on his website https://inventwithpython.com/. His cat now weighs 11 pounds.

About the Tech Reviewer Philip James has been working in Python for over a decade and is a frequent speaker in the Python community. He speaks on topics ranging from Unix fundamentals to open source social networks. Philip is a Core Contributor to the BeeWare project and lives in the San Francisco Bay Area with his partner Nic and her cat River.

BRIEF CONTENTS Acknowledgments Introduction PART I: PYTHON PROGRAMMING BASICS Chapter 1: Python Basics Chapter 2: Flow Control Chapter 3: Functions Chapter 4: Lists Chapter 5: Dictionaries and Structuring Data Chapter 6: Manipulating Strings PART II: AUTOMATING TASKS Chapter 7: Pattern Matching with Regular Expressions Chapter 8: Input Validation Chapter 9: Reading and Writing Files Chapter 10: Organizing Files Chapter 11: Debugging Chapter 12: Web Scraping Chapter 13: Working with Excel Spreadsheets Chapter 14: Working with Google Sheets Chapter 15: Working with PDF and Word Documents Chapter 16: Working with CSV Files and JSON Data

Chapter 17: Keeping Time, Scheduling Tasks, and Launching Programs Chapter 18: Sending Email and Text Messages Chapter 19: Manipulating Images Chapter 20: Controlling the Keyboard and Mouse with GUI Automation Appendix A: Installing Third-Party Modules Appendix B: Running Programs Appendix C: Answers to the Practice Questions Index

CONTENTS IN DETAIL ACKNOWLEDGMENTS INTRODUCTION Whom Is This Book For? Conventions What Is Programming? What Is Python? Programmers Don’t Need to Know Much Math You Are Not Too Old to Learn Programming Programming Is a Creative Activity About This Book Downloading and Installing Python Downloading and Installing Mu Starting Mu Starting IDLE The Interactive Shell Installing Third-Party Modules How to Find Help Asking Smart Programming Questions Summary PART I: PYTHON PROGRAMMING BASICS 1 PYTHON BASICS Entering Expressions into the Interactive Shell The Integer, Floating-Point, and String Data Types String Concatenation and Replication Storing Values in Variables

Assignment Statements Variable Names Your First Program Dissecting Your Program Comments The print() Function The input() Function Printing the User’s Name The len() Function The str(), int(), and float() Functions Summary Practice Questions 2 FLOW CONTROL Boolean Values Comparison Operators Boolean Operators Binary Boolean Operators The not Operator Mixing Boolean and Comparison Operators Elements of Flow Control Conditions Blocks of Code Program Execution Flow Control Statements if Statements else Statements elif Statements while Loop Statements break Statements continue Statements for Loops and the range() Function

Importing Modules from import Statements Ending a Program Early with the sys.exit() Function A Short Program: Guess the Number A Short Program: Rock, Paper, Scissors Summary Practice Questions 3 FUNCTIONS def Statements with Parameters Define, Call, Pass, Argument, Parameter Return Values and return Statements The None Value Keyword Arguments and the print() Function The Call Stack Local and Global Scope Local Variables Cannot Be Used in the Global Scope Local Scopes Cannot Use Variables in Other Local Scopes Global Variables Can Be Read from a Local Scope Local and Global Variables with the Same Name The global Statement Exception Handling A Short Program: Zigzag Summary Practice Questions Practice Projects The Collatz Sequence Input Validation 4 LISTS The List Data Type

Getting Individual Values in a List with Indexes Negative Indexes Getting a List from Another List with Slices Getting a List’s Length with the len() Function Changing Values in a List with Indexes List Concatenation and List Replication Removing Values from Lists with del Statements Working with Lists Using for Loops with Lists The in and not in Operators The Multiple Assignment Trick Using the enumerate() Function with Lists Using the random.choice() and random.shuffle() Functions with Lists Augmented Assignment Operators Methods Finding a Value in a List with the index() Method Adding Values to Lists with the append() and insert() Methods Removing Values from Lists with the remove() Method Sorting the Values in a List with the sort() Method Reversing the Values in a List with the reverse() Method Example Program: Magic 8 Ball with a List Sequence Data Types Mutable and Immutable Data Types The Tuple Data Type Converting Types with the list() and tuple() Functions References Identity and the id() Function Passing References The copy Module’s copy() and deepcopy() Functions A Short Program: Conway’s Game of Life Summary Practice Questions

Practice Projects Comma Code Coin Flip Streaks Character Picture Grid 5 DICTIONARIES AND STRUCTURING DATA The Dictionary Data Type Dictionaries vs. Lists The keys(), values(), and items() Methods Checking Whether a Key or Value Exists in a Dictionary The get() Method The setdefault() Method Pretty Printing Using Data Structures to Model Real-World Things A Tic-Tac-Toe Board Nested Dictionaries and Lists Summary Practice Questions Practice Projects Chess Dictionary Validator Fantasy Game Inventory List to Dictionary Function for Fantasy Game Inventory 6 MANIPULATING STRINGS Working with Strings String Literals Indexing and Slicing Strings The in and not in Operators with Strings Putting Strings Inside Other Strings Useful String Methods The upper(), lower(), isupper(), and islower() Methods

The isX() Methods The startswith() and endswith() Methods The join() and split() Methods Splitting Strings with the partition() Method Justifying Text with the rjust(), ljust(), and center() Methods Removing Whitespace with the strip(), rstrip(), and lstrip() Methods Numeric Values of Characters with the ord() and chr() Functions Copying and Pasting Strings with the pyperclip Module Project: Multi-Clipboard Automatic Messages Step 1: Program Design and Data Structures Step 2: Handle Command Line Arguments Step 3: Copy the Right Phrase Project: Adding Bullets to Wiki Markup Step 1: Copy and Paste from the Clipboard Step 2: Separate the Lines of Text and Add the Star Step 3: Join the Modified Lines A Short Progam: Pig Latin Summary Practice Questions Practice Projects Table Printer Zombie Dice Bots PART II: AUTOMATING TASKS 7 PATTERN MATCHING WITH REGULAR EXPRESSIONS Finding Patterns of Text Without Regular Expressions Finding Patterns of Text with Regular Expressions Creating Regex Objects Matching Regex Objects Review of Regular Expression Matching

More Pattern Matching with Regular Expressions Grouping with Parentheses Matching Multiple Groups with the Pipe Optional Matching with the Question Mark Matching Zero or More with the Star Matching One or More with the Plus Matching Specific Repetitions with Braces Greedy and Non-greedy Matching The findall() Method Character Classes Making Your Own Character Classes The Caret and Dollar Sign Characters The Wildcard Character Matching Everything with Dot-Star Matching Newlines with the Dot Character Review of Regex Symbols Case-Insensitive Matching Substituting Strings with the sub() Method Managing Complex Regexes Combining re.IGNORECASE, re.DOTALL, and re.VERBOSE Project: Phone Number and Email Address Extractor Step 1: Create a Regex for Phone Numbers Step 2: Create a Regex for Email Addresses Step 3: Find All Matches in the Clipboard Text Step 4: Join the Matches into a String for the Clipboard Running the Program Ideas for Similar Programs Summary Practice Questions Practice Projects Date Detection Strong Password Detection Regex Version of the strip() Method

8 INPUT VALIDATION The PyInputPlus Module The min, max, greaterThan, and lessThan Keyword Arguments The blank Keyword Argument The limit, timeout, and default Keyword Arguments The allowRegexes and blockRegexes Keyword Arguments Passing a Custom Validation Function to inputCustom() Project: How to Keep an Idiot Busy for Hours Project: Multiplication Quiz Summary Practice Questions Practice Projects Sandwich Maker Write Your Own Multiplication Quiz 9 READING AND WRITING FILES Files and File Paths Backslash on Windows and Forward Slash on macOS and Linux Using the / Operator to Join Paths The Current Working Directory The Home Directory Absolute vs. Relative Paths Creating New Folders Using the os.makedirs() Function Handling Absolute and Relative Paths Getting the Parts of a File Path Finding File Sizes and Folder Contents Modifying a List of Files Using Glob Patterns Checking Path Validity The File Reading/Writing Process

Opening Files with the open() Function Reading the Contents of Files Writing to Files Saving Variables with the shelve Module Saving Variables with the pprint.pformat() Function Project: Generating Random Quiz Files Step 1: Store the Quiz Data in a Dictionary Step 2: Create the Quiz File and Shuffle the Question Order Step 3: Create the Answer Options Step 4: Write Content to the Quiz and Answer Key Files Project: Updatable Multi-Clipboard Step 1: Comments and Shelf Setup Step 2: Save Clipboard Content with a Keyword Step 3: List Keywords and Load a Keyword’s Content Summary Practice Questions Practice Projects Extending the Multi-Clipboard Mad Libs Regex Search 10 ORGANIZING FILES The shutil Module Copying Files and Folders Moving and Renaming Files and Folders Permanently Deleting Files and Folders Safe Deletes with the send2trash Module Walking a Directory Tree Compressing Files with the zipfile Module Reading ZIP Files Extracting from ZIP Files Creating and Adding to ZIP Files

Project: Renaming Files with American-Style Dates to European-Style Dates Step 1: Create a Regex for American-Style Dates Step 2: Identify the Date Parts from the Filenames Step 3: Form the New Filename and Rename the Files Ideas for Similar Programs Project: Backing Up a Folder into a ZIP File Step 1: Figure Out the ZIP File’s Name Step 2: Create the New ZIP File Step 3: Walk the Directory Tree and Add to the ZIP File Ideas for Similar Programs Summary Practice Questions Practice Projects Selective Copy Deleting Unneeded Files Filling in the Gaps 11 DEBUGGING Raising Exceptions Getting the Traceback as a String Assertions Using an Assertion in a Traffic Light Simulation Logging Using the logging Module Don’t Debug with the print() Function Logging Levels Disabling Logging Logging to a File Mu’s Debugger Continue Step In

Step Over Step Out Stop Debugging a Number Adding Program Breakpoints Summary Practice Questions Practice Project Debugging Coin Toss 12 WEB SCRAPING Project: mapIt.py with the webbrowser Module Step 1: Figure Out the URL Step 2: Handle the Command Line Arguments Step 3: Handle the Clipboard Content and Launch the Browser Ideas for Similar Programs Downloading Files from the Web with the requests Module Downloading a Web Page with the requests.get() Function Checking for Errors Saving Downloaded Files to the Hard Drive HTML Resources for Learning HTML A Quick Refresher Viewing the Source HTML of a Web Page Opening Your Browser’s Developer Tools Using the Developer Tools to Find HTML Elements Parsing HTML with the bs4 Module Creating a BeautifulSoup Object from HTML Finding an Element with the select() Method Getting Data from an Element’s Attributes Project: Opening All Search Results

Step 1: Get the Command Line Arguments and Request the Search Page Step 2: Find All the Results Step 3: Open Web Browsers for Each Result Ideas for Similar Programs Project: Downloading All XKCD Comics Step 1: Design the Program Step 2: Download the Web Page Step 3: Find and Download the Comic Image Step 4: Save the Image and Find the Previous Comic Ideas for Similar Programs Controlling the Browser with the selenium Module Starting a selenium-Controlled Browser Finding Elements on the Page Clicking the Page Filling Out and Submitting Forms Sending Special Keys Clicking Browser Buttons More Information on Selenium Summary Practice Questions Practice Projects Command Line Emailer Image Site Downloader 2048 Link Verification 13 WORKING WITH EXCEL SPREADSHEETS Excel Documents Installing the openpyxl Module Reading Excel Documents Opening Excel Documents with OpenPyXL