Modern Python Cookbook, Third Edition 130+ updated recipes for modern Python 3.12 with new techniques and tools (Steven F. Lott) (z-library.sk, 1lib.sk, z-lib.sk)

Modern Python Cookbook Third Edition 130+ updated recipes for modern Python 3.12 with new techniques and tools Steven F. Lott Packt and this book are not officially connected with Python. This book is an effort from the Python community of experts to help more developers.

Modern Python Cookbook Third Edition Copyright © 2024 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Senior Publishing Product Manager: Denim Pinto Acquisition Editor – Peer Reviews: Swaroop Singh Project Editor: Parvathy Nair Senior Development Editor: Elliot Dallow Copy Editor: Safis Editing Technical Editor: Karan Sonawane Proofreader: Safis Editing Indexer: Subalakshmi Govindhan Presentation Designer: Ganesh Bhadwalkar Developer Relations Marketing Executive: Vignesh Raju First published: November 2016 Second edition: July 2020 Third edition: July 2024 Production reference: 1290724 Published by Packt Publishing Ltd. Grosvenor House 11 St Paul’s Square Birmingham B3 1RB, UK. ISBN 978-1-83546-638-4 www.packt.com

Contributors About the author Steven F. Lott has been turning coffee into software since the days when computers were large, expensive, and rare. Working for decades in high tech has given him exposure to a lot of ideas and techniques, some bad, but most are useful and helpful to others. Steven has been working with Python since the ‘90s, building a variety of tools and appli- cations. He’s written a number of books, and sometimes manages to attend conferences. Steven is currently a tech nomad who lives in various places on the East Coast of the US. He tries to live by the words, “Don’t come home until you have a story.” You can find him online at https://fosstodon.org/@slott56.

About the reviewers Abhijeet Jagutai Dada Mote is a software engineer with expertise in data engineering, data analysis, data science, and automation. He has worked in AdTech, semiconductors, IoT, and media industries and is currently working on a cutting-edge CTV ad platform. He is proficient in Python, development of automation testing frameworks, advanced big data pipelines, and AI-driven anomaly detection systems. He has also contributed to smart city projects, holds various AI related patents, is working on several AI research papers, and delivered a workshop at PyCon Italia. Find out more at: https://scholar.google.com/c itations?user=v4tFOAgAAAAJ I would like to thank my wife, Priyanka, and our one-year-old child, Advaith, for their support and patience. I also thank my Aai-Baba, friends, and colleagues who have helped me become who I am today. Francisco Benavides has over 20 years of experience in programming, with the last 10 focused on Python development for ETL server applications. He has been part of worldwide teams spanning five continents, has been a team leader and specialist in the telecom sector, a Python developer, and has now been a data engineer in the energy sector for over 5 years, where he has developed a multitude of data ingestion platforms using Linux, Azure, and AWS. I thank God for the opportunities he has allowed me to take and a special thank you for the blessings I received from my wife and kids, who have endured, at times, long months apart, working and encouraging me to keep going. I also thank the editorial team for trusting me and allowing me to collaborate in the making of this book.

Join our community Discord space Join our Python Discord workspace to discuss and find out more about the book: https://packt.link/dHrHU

Table of Contents Preface xv Chapter 1: Numbers, Strings, and Tuples 1 Choosing between float, decimal, and fraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Choosing between true division and floor division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 String parsing with regular expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Building complicated strings with f-strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Building complicated strings from lists of strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Using the Unicode characters that aren’t on our keyboards . . . . . . . . . . . . . . . . . . . . . . . . . 26 Encoding strings – creating ASCII and UTF-8 bytes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Decoding bytes – how to get proper characters from some bytes . . . . . . . . . . . . . . . . . . . 33 Using tuples of items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Using NamedTuples to simplify item access in tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Chapter 2: Statements and Syntax 45 Writing Python script and module files – syntax basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Writing long lines of code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Including descriptions and documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Writing better docstrings with RST markup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Designing complex if...elif chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Saving intermediate results with the := “walrus” operator . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Avoiding a potential problem with break statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Leveraging exception matching rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Avoiding a potential problem with an except: clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Concealing an exception root cause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Managing a context using the with statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Chapter 3: Function Definitions 97 Function parameters and type hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Designing functions with optional parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Using super flexible keyword parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Forcing keyword-only arguments with the * separator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Defining position-only parameters with the / separator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Picking an order for parameters based on partial functions . . . . . . . . . . . . . . . . . . . . . . . . . 122 Writing clear documentation strings with RST markup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Designing recursive functions around Python’s stack limits . . . . . . . . . . . . . . . . . . . . . . . . . 134 Writing testable scripts with the script-library switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Chapter 4: Built-In Data Structures Part 1: Lists and Sets 145 Choosing a data structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Building lists – literals, appending, and comprehensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Slicing and dicing a list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Shrinking lists – deleting, removing, and popping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Writing list-related type hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Reversing a copy of a list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Building sets – literals, adding, comprehensions, and operators . . . . . . . . . . . . . . . . . . . . . 177 Shrinking sets – remove(), pop(), and difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Writing set-related type hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Chapter 5: Built-In Data Structures Part 2: Dictionaries 195 Creating dictionaries – inserting and updating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Shrinking dictionaries – the pop() method and the del statement . . . . . . . . . . . . . . . . . . . 202 Writing dictionary-related type hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Understanding variables, references, and assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Making shallow and deep copies of objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Avoiding mutable default values for function parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Chapter 6: User Inputs and Outputs 229 Using the features of the print() function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 Using input() and getpass() for user input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 Debugging with f“{value=}” strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Using argparse to get command-line input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Using invoke to get command-line input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Using cmd to create command-line applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Using the OS environment settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Chapter 7: Basics of Classes and Objects 263 Using a class to encapsulate data and processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Essential type hints for class definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Designing classes with lots of processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 Using typing.NamedTuple for immutable objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Using dataclasses for mutable objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Using frozen dataclasses for immutable objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Optimizing small objects with __slots__ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Using more sophisticated collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Extending a built-in collection – a list that does statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Using properties for lazy attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Creating contexts and context managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Managing multiple contexts with multiple resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 Chapter 8: More Advanced Class Design 327

Choosing between inheritance and composition – the “is-a” question . . . . . . . . . . . . . . 328 Separating concerns via multiple inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Leveraging Python’s duck typing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 Managing global and singleton objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Using more complex structures – maps of lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 Creating a class that has orderable objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 Deleting from a list of complicated objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364 Chapter 9: Functional Programming Features 371 Writing generator functions with the yield statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 Applying transformations to a collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Using stacked generator expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 Picking a subset – three ways to filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 Summarizing a collection – how to reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Combining the map and reduce transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Implementing “there exists” processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 Creating a partial function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Writing recursive generator functions with the yield from statement . . . . . . . . . . . . . . . 423 Chapter 10: Working with Type Matching and Annotations 431 Designing with type hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Using the built-in type matching functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Using the match statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 Handling type conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 Implementing more strict type checks with Pydantic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Including run-time valid value checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 Chapter 11: Input/Output, Physical Format, and Logical Layout 471 Using pathlib to work with filenames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 Replacing a file while preserving the previous version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

Reading delimited files with the CSV module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488 Using dataclasses to simplify working with CSV files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 Reading complex formats using regular expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500 Reading JSON and YAML documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508 Reading XML documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 Reading HTML documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523 Chapter 12: Graphics and Visualization with Jupyter Lab 533 Starting a Notebook and creating cells with Python code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 Ingesting data into a notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540 Using pyplot to create a scatter plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546 Using axes directly to create a scatter plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552 Adding details to markdown cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 Including Unit Test Cases in a Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562 Chapter 13: Application Integration: Configuration 567 Finding configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 Using TOML for configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 Using Python for configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580 Using a class as a namespace for configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585 Designing scripts for composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 Using logging for control and audit output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596 Chapter 14: Application Integration: Combination 605 Combining two applications into one . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606 Combining many applications using the Command design pattern . . . . . . . . . . . . . . . . 614 Managing arguments and configuration in composite applications . . . . . . . . . . . . . . . . . 620 Wrapping and combining CLI applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Wrapping a program and checking the output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632

Chapter 15: Testing 641 Using docstrings for testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Testing functions that raise exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651 Handling common doctest issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654 Unit testing with the unittest module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661 Combining unittest and doctest tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667 Unit testing with the pytest module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672 Combining pytest and doctest tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677 Testing things that involve dates or times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681 Testing things that involve randomness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686 Mocking external resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693 Chapter 16: Dependencies and Virtual Environments 703 Creating environments using the built-in venv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706 Installing packages with a requirements.txt file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711 Creating a pyproject.toml file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716 Using pip-tools to manage the requirements.txt file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723 Using Anaconda and the conda tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 727 Using the poetry tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732 Coping with changes in dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736 Chapter 17: Documentation and Style 743 The bare minimum: a README.rst file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744 Installing Sphinx and creating documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 749 Using Sphinx autodoc to create the API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754 Identifying other CI/CD tools in pyproject.toml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 760 Using tox to run comprehensive quality checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764 Other Books You May Enjoy 773

Index 777

Preface Python is the preferred language choice of developers, engineers, data scientists, and hobbyists everywhere. It is a great scripting language that can power your applications and provide speed, safety, and scalability. By exposing Python as a series of simple recipes, this book can help you gain insights into specific language features in a concrete context. The idea is to avoid abstract discussions of language features and focus on applying the language to concrete data and processing problems. What you need for this book All you need to follow through the examples in this book is a computer running any Python, version 3.12 or newer. Many of the examples can be adapted to work with Python 3 versions prior to 3.12. Material in Chapter 10 describes the match statement, introduced with Python 3.10. We strongly encourage installing a fresh copy of Python, avoiding any pre-installed operat- ing system Python. The language run-time can be downloaded from https://www.python.org/downloads/. An alternative is to start with the Miniconda tool (https://docs.conda.io/en/latest/miniconda.html) and use conda to create a Python 3.12 (or newer) environment. Python 2 cannot be used anymore. Since 2020, Python 2 is no longer an alternative.

xvi Preface Who this book is for The book is for web developers, programmers, enterprise programmers, engineers, and big data scientists. If you are a beginner, this book can get you started. If you are experienced, it will expand your knowledge base. A basic knowledge of programming will help; while some foundational topics are covered, this is not a tutorial on programming or Python. What this book covers There are over 130 recipes in this book. We can decompose them into four general areas: • Python Fundamentals Chapter 1, Numbers, Strings, and Tuples, will look at the different kinds of numbers, how to work with strings, how to use tuples, and how to use the essential built-in types in Python. We will also show ways to exploit the full power of the Unicode character set. Chapter 2, Statements and Syntax, will cover some basics of creating script files. Then we’ll move on to looking at some of the complex statements, including if, while, for, break, try, raise, and with. Chapter 3, Function Definitions, will look at a number of function definition techniques. We’ll devote several recipes to type hints for a variety of types. We’ll also address an element of designing a testable script by using functions and a main-import-switch. Chapter 4, Built-In Data Structures Part 1: Lists and Sets, begins an overview of the built-in data structures structures that are available and what problems they solve. This includes a number of recipes showing list and set operations, including list and set comprehensions. Chapter 5, Built-In Data Structures Part 2: Dictionaries, continues examining the built-in data structures, looking at dictionaries in detail. This chapter will also look at some more advanced topics related to how Python handles references to objects. It also shows how to handle mutable objects as function parameter default values.

Preface xvii Chapter 6, User Inputs and Outputs, explains how to use the different features of the print() function. We’ll also look at the different functions used to provide user input. The use of f-strings for debugging and the argparse module for command-line input are featured. • Object-Oriented and Functional Design Approaches Chapter 7, Basics of Classes and Objects, begins the coverage of object-oriented programming. It shows how to create classes and the type hints related to class definitions. This section has been expanded from previous editions to cover data- classes. It shows how to extend built-in classes, and how to create context managers to manage resources. Chapter 8, More Advanced Class Design, continues the exploration of object-oriented design and programming. This includes an exploration of the composition vs. inheri- tance question, and shows how to manage the “duck typing” principle of Python. Chapter 9, Functional Programming Features, looks at Python’s functional program- ming features. This style of programming emphasizes function definitions and stateless, immutable objects. The recipes look at generator expressions, using the map(), filter(), and reduce() functions. We also look at ways to create partial functions and some examples of replacing stateful objects with data structures built from collections of immutable objects. • More Sophisticated Designs Chapter 10, Working with Type Matching and Annotations, looks more closely at type hints and the match statement. This includes using Pydantic to create classes with more strict run-time type-checking. It also looks at introspection of annotated types. Chapter 11, Input/Output, Physical Format, and Logical Layout, will work with paths and files in general. It will look at reading and writing data in a variety of file formats, including CSV, JSON (and YAML), XML, and HTML. The HTML section will emphasize using Beautiful Soup for extracting data.

xviii Preface Chapter 12, Graphics and Visualization with Jupyter Lab, will use Jupyter Lab to create notebooks that use Python for data analysis and visualization. This will show ways to ingest data into a notebook to create plots, and how to use Markdown to create useful documentation and reports from a notebook. Chapter 13, Application Integration: Configuration, will start looking at ways that we can design larger applications. The recipes in this chapter address different ways to handle configuration files and how to manage logging. Chapter 14, Application Integration: Combination, will continue looking at ways to create composite applications from smaller pieces. This will look at object-oriented design patterns and Command-Line Interface (CLI) applications. It will also look at using the subprocess module to run existing applications under Python’s control. • Completing a Project: Fit and Finish Chapter 15, Testing, provides recipes for using the built-in doctest and unittest testing frameworks used in Python. Additionally, recipes will cover the pytest tool. Chapter 16, Dependencies and Virtual Environments, covers tools used to manage virtual environments. The built-in venv, as well as conda and poetry will be covered. There are a lot of solutions to managing virtual environments, and we can’t cover all of them. Chapter 17, Documentation and Style, covers additional tools that can help to create high-quality software. This includes a particular focus on sphinx for creating comprehensive, readable documentation. We’ll also look at tox to automate running tests. To get the most out of this book To get the most out of this book you can download the example code files and the color images as per the instructions below.

Preface xix Download the example code files The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Modern-Python-Cookbook-Third-Edition. This repository is also the best place to start a conversation about specific topics discussed in the book. Feel free to open an issue if you want to engage with the authors or other readers. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out! Download the color images We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://packt.link/gbp/9781835466384. Conventions used In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning. Code words in text, database table names, folder names, filenames, file extensions, path- names, dummy URLs, user input, and Twitter handles are shown as follows: “We can include other contexts through the use of the include directive.” A block of code is set as follows: if distance is None: distance = rate * time elif rate is None: rate = distance / time elif time is None: time = distance / rate Any command-line input or output is written as follows: >>> import math >>> math.factorial(52) 80658175170943878571660636856403766975289505440883277824000000000000