Speed up your Django tests (Adam Johnson) (Z-Library)

📄 File Format: PDF

💾 File Size: 5.7 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Adam Johnson

📄 Page 2

© 2020 Adam Johnson. All rights reserved. Published by Adam’s Web Services Ltd, UK. For recent changes see the changelog section. The moral right of the author has been asserted. See my website at https://adamj.eu for contact details and more information. Written in British English using Oxford spelling, so it’s ”behaviour” with a ”u” and ”opti- mize” with a ”z”. Created with Sphinx. Typeset in PT Sans and Ubuntu Monospace. Front cover illustration by katerinadot on Fiverr. Other illustrations from British Library on Flickr, The Internet Archive on Flickr, or self- made with GIMP and Inkscape. The only authorized vendor or distributor for this product is adamchainz on Gumroad.

📄 Page 3

Contents 1 Introduction 1 1.1 Who Is This Book For? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 A Brief Tour of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.5 Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Toolbox 9 2.1 Test Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Custom test Management Command . . . . . . . . . . . . . . . . . . . 13 2.3 Test Runner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.4 Test Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5 Custom TestCase Classes . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6 Third-Party Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3 Measure! 27 3.1 Built-in Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2 Shell Timing Commands . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.3 Measure Individual Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 Pro le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4 Easy Wins 46 4.1 Use a Faster Password Hasher . . . . . . . . . . . . . . . . . . . . . . . 47 4.2 Always Rebuild the Test Database if It Exists . . . . . . . . . . . . . . 47 4.3 Disable Database Serialization . . . . . . . . . . . . . . . . . . . . . . . 49 4.4 Disable Instrumentation Packages during Tests . . . . . . . . . . . . . 50 4.5 Use an In-Memory File Storage Backend . . . . . . . . . . . . . . . . . 52 4.6 Use an In-Memory Cache Backend . . . . . . . . . . . . . . . . . . . . 53 4.7 Use In-Memory Backends for your Task Queues . . . . . . . . . . . . . 57 4.8 Skip Slow Tests Locally . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.9 Prevent Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 i

📄 Page 4

4.10 Prevent Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.11 Reduce pytest Test Collection . . . . . . . . . . . . . . . . . . . . . . . 69 4.12 Prevent WhiteNoise From Scanning All Static Files . . . . . . . . . . . 70 5 Upgrades 71 5.1 Upgrade Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.2 Upgrade Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.3 Upgrade Your Database Server . . . . . . . . . . . . . . . . . . . . . . . 77 6 Parallelize 79 6.1 What Is Parallel Testing? . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.2 How to Move To Parallel Testing . . . . . . . . . . . . . . . . . . . . . . 80 6.3 Check Your Tests Are Isolated . . . . . . . . . . . . . . . . . . . . . . . 81 6.4 Activate Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6.5 Deal with Shared Resources . . . . . . . . . . . . . . . . . . . . . . . . 91 6.6 Split Up Large Test Groups . . . . . . . . . . . . . . . . . . . . . . . . . 97 7 Migrations 100 7.1 Reuse the Test Database Between Runs . . . . . . . . . . . . . . . . . 101 7.2 Squash Your Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 7.3 Don’t Disable Migrations in Tests . . . . . . . . . . . . . . . . . . . . . 105 8 Database Con guration 110 8.1 Use In-Memory Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 8.2 Don’t Swap Your Database to SQLite in Tests . . . . . . . . . . . . . . 115 9 CI Con guration 117 9.1 Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 9.2 Scale Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 9.3 Parallelize Across Multiple CI Servers . . . . . . . . . . . . . . . . . . . 121 10 Test Structure 124 10.1 AAA: Arrange-Act-Assert . . . . . . . . . . . . . . . . . . . . . . . . . . 125 10.2 Write Mostly Unit Tests Not Integration Tests . . . . . . . . . . . . . . 129 10.3 Use the Right TestCase Class . . . . . . . . . . . . . . . . . . . . . . . 141 10.4 TestCase Transaction Blockers . . . . . . . . . . . . . . . . . . . . . . . 144 11 Test Data 151 11.1 Avoid Fixture Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 11.2 Avoid Common Data in Custom TestCase Classes . . . . . . . . . . . . 153 11.3 Use Factories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 11.4 Use setUpTestData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 11.5 Optimize Database Queries . . . . . . . . . . . . . . . . . . . . . . . . . 162 ii

📄 Page 5

11.6 Advanced TestCase Features . . . . . . . . . . . . . . . . . . . . . . . 163 12 Targeted Mocking 166 12.1 The Five Kinds of Mock Objects . . . . . . . . . . . . . . . . . . . . . . 166 12.2 The Danger With Mocking . . . . . . . . . . . . . . . . . . . . . . . . . 168 12.3 unittest.mock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 12.4 Mock Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 12.5 Mock Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 12.6 Mock Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 12.7 Mock HTTP Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 12.8 Mock Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 13 Outroduction 192 13.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 13.2 Thank You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 iii

📄 Page 6

Chapter 1 Introduction The measure of success most often is speed. Doing things better is synony- mous with doing things faster so that we can do even more things ef ciently and effectively. In buying into this premise, we enter a spiral of acceleration that we can never hope to master. —Diana Scharf Hunt and Pam Hait Welcome, dear reader. Oh, tests. Tests are great. Tests show us where bugs are… at least some of them. Tests help us to not add new bugs. But… tests. Tests are also a frequent source of frustration. When you rst started writing tests, you probably found them an extra nuisance to write. You probably still think the same, at least sometimes. Even if you love writing tests, running them can be boring. Slow tests can make you lose concentration, juggle tasks, and end up getting less stuff done. They can suck all the fun 1

📄 Page 7

Speed Up Your Django Tests, Release 2021-07-08 out of programming. And programming is meant to be fun! There’s a phase shift when you speed up your tests, restoring that joy. If your tests take 1000 seconds and you reduce them to 600 seconds, you’ll feel it. Your whole team will feel it. The same happens if you can reduce a test run from 30 seconds down to 10 seconds. And there’s no limit to “fast enough”. Reducing test run time is the easiest and safest way to increase your speed of delivery. And your organization wants to deliver as fast as possible, to get valuable features and xes in front of users. Fast tests also tend to bemore accurate tests. They’re well targeted, so they only fail when a feature is broken. And with less overhead, you can exercise features more completely. The content is based on my eight years of experience with tests on Django projects. I’ve sped upmany projects’ test suites, improved Django’s own testing framework, and created several pytest plugins. In your hands, or at least on your screen, is my best guide for speeding up tests on your project. I’ve tried to cover most situations, so you should be able to nd something relevant to your project. May it reduce your test speed and your test pain. Enjoy! —Adam P.S. Please send me any and all feedback through the contact details on my site1. 1.1 Who Is This Book For? If you are a Django developer working with tests, this is for you. No matter your level. My aim is to make the book readable by a junior developer who has been given the vague task of “improving the tests”, but also to include material that even long time Django users may not have come across. I’ve tried to be opinionated enough to be actionable, but provide advice that’s applicable to all projects. 1 https://adamj.eu/contact/ 2 Chapter 1. Introduction

📄 Page 8

Speed Up Your Django Tests, Release 2021-07-08 1.2 A Brief Tour of This Book The rst two chapters, Toolbox and Measure!, explain the various tools you have at hand for modifying and measuring your tests respectively. The next six chapters cover ways of speeding up your tests without the bother of rewrit- ing your test code (much). Easy Wins covers a number of smaller, non-invasive changes that can speed up your tests. Upgrades then re-iterates the importance of keeping things upgraded, especially with respect to test performance. Parallelize covers using paral- lelized testing, and the challenges you might face adding it. Migrations covers tactics for improving your work ow with database migrations. Finally, Database Con guration and CI Con guration cover ways of speeding up your tests in con guration for those environ- ments. The three chapters that follow cover how to write fast tests. Test Structure and Test Data go into depth on writing faster tests that use less data. And Targeted Mocking shows some focussed techniques for replacing components during testing that can help you write better, faster tests. Finally, the Outroduction gives some closing remarks and links to further reading. 1.2.1 In a rush? To get the 80% of improvements that take 20% of the effort: • Record your current test runtime with a shell timing command as in Measure!. • Skim Easy Wins for any changes that are applicable to your project and make them. When necessary, follow the references back to Toolbox to understand the pieces you’re changing. • If you don’t yet run your tests in parallel, read Parallelize. • Learn how to reuse your database and squash your migrations in Migrations. • Do as many of the upgrades described in Upgrades as possible. • Compare your test runtime! If after this, your tests are still too slow, pro le them with one of the tools described in the second half ofMeasure!. Then when you know where to focus, read the later chapters to determine ways of speeding up the slowest tests. 1.2. A Brief Tour of This Book 3

📄 Page 9

Speed Up Your Django Tests, Release 2021-07-08 1.3 Examples This book has a lot of code examples. I hope that is not a surprise! Here’s some information on their layout. 1.3.1 Commands Commands to run in your shell look like this: $ python --version The $ represents the prompt - don’t type that. I’m using macOS but hope to have removed any macOS or *nix bias. Commands should work cross-platform, but where that’s not possible I’ve added a separate Windows example with a PowerShell prompt: > python --version If I’ve written something incompatible with your platform (most likely Windows), please let me know. 1.3.2 Versions Examples have been prepared with Python 3.9 and Django 3.2. I’m only using the of cial Python distribution, CPython, since Django or its dependencies often don’t work on other Python interpreters. Python code is formatted with Black2. (Like Django itself will be one day3.) Import statements are sorted and grouped with isort4. And all code is also linted with Flake85. Throughout, I refer to installation commands with: $ python -m pip install Django I’m using python -m pip as per pip developers’ recommendations - see my blog post on the subject6. 2 https://black.readthedocs.io/en/stable/ 3 https://github.com/django/deps/blob/main/accepted/0008-black.rst 4 https://pypi.org/project/isort/ 5 https://flake8.pycqa.org/en/latest/index.html 6 https://adamj.eu/tech/2020/02/25/use-python-m-pip-everywhere/ 4 Chapter 1. Introduction

📄 Page 10

Speed Up Your Django Tests, Release 2021-07-08 Please read these commands as “install using your dependency management solution”. There are several tools for this in the Python ecosystem, and I can’t cover them all. If you aren’t using any dependencymanagement tool (beyond pip), pip-compile7 is a great start as it’s a small layer on top of pip. It’s also my preference. 1.3.3 Django Projects The example projects I’ve used are based on my “simple-core” startproject template8. This has directory structure like this: $ tree . ├── example │ ├── __init__.py │ ├── core │ │ ├── __init__.py │ │ ├── models.py │ │ └── tests │ │ ├── __init__.py │ │ └── test_models.py │ ├── settings.py │ └── urls.py └── manage.py 3 directories, 8 files This is slightly different to Django’s normal startproject template. The whole project lives inside a single Python package, example. example.core is an app, inside which there are models and tests. This is counter to the Django default of having apps in modules that are peers to the “project app”. I prefer this layout because it makes it clear when you’re importing something belonging to your project. Owned names always start with example (or whatever your project is called). 7 https://pypi.org/project/pip-tools/ 8 https://github.com/adamchainz/django-startproject-templates 1.3. Examples 5

📄 Page 11

Speed Up Your Django Tests, Release 2021-07-08 1.3.4 resources.zip This PDF also came with a resources.zip. This includes all the example projects. They’re structured with one directory per chapter, within which there are several indi- vidual projects. When a le would be particularly useful to open, I mention the le name to open. Oth- erwise, the code examples don’t refer to their le names, since they should stand alone. If you do nd yourself wanting to refer to some code, the les should be named well enough to gure out which one you want. 1.3.5 Snipped Output Quite often in examples, I “snip” the code or output to keep it short and discard irrelevant details. You can spot this whenever there’s a .... For example, here I’ve snipped the list of subcommands: $ python manage.py --help Type 'manage.py help <subcommand>' for help on a specific subcommand. Available subcommands: [django] check ... testserver In Python, ... is also syntactically valid - it’s the rarely needed Ellipsis object9. This never has any meaning in my examples. 1.4 Acknowledgements Thank you to everyone who helped me put this book together. Thanks to my partner Mafalda for her love, encouraging me to carry on when I needed it, and encouraging me to stop at the end of the day. Thanks to my parents - my mum for sharing her experience as an author, and my dad for proofreading this book with the same care he applied to every homework essay and CV. And thanks to my brother and sister-in-law for their support. 9 https://docs.python.org/3/library/stdtypes.html#bltin-ellipsis-object 6 Chapter 1. Introduction

📄 Page 12

Speed Up Your Django Tests, Release 2021-07-08 I was very happy to have three excellent friends offer their help as reviewers. You should check out their blogs! Thanks to David Seddon10 for helping organize the London Django Meetup, giving many great testing and design talks there, and balancing my Aristotelian tendencies with his Platonist ones. Thanks to Jeroen Janssens11 for letting me deploy his recommendation system, letting me crash on the way to African Python conferences, and having an eye for both design and wording. And thanks to Julius Šėporaitis12 for pushing for a culture of excellence every time I’ve worked with him, bringing real magic everywhere he goes, and suggesting a broad range of enhancements. I’d also like to thank these people for their support and feedback: Ahter Sönmez, Carlton Gibson, Dan Palmer, Gordon Wrigley, James Cooke, Jeff Triplett, Kristian Glass, Marco Al- abruzzo, Mariusz Felisiak, Peter Inglesby, Rupert Baker, Tom Grainger, Will Vincent, Çağıl Uluşahin, Matt Layman. And Kevin the puppy. And nally, thank you reader for purchasing my book. I hope you nd it useful. 1.5 Changelog 2021-07-08 • Added ePub le to the release. Thanks to Mike Lissner and Paolo Melchiorre for requesting and testing it. • Upgraded to Python 3.9 and Django 3.2 and updated content to re ect changes. • Added Mock Time section to the “Targeted Mocking” chapter, describing my library time-machine. • Added a section to “Parallelize” on implementing shared resource locking with a pytest xture. • Added note in Chapter 9 about Circle CI test splitting. • Updated section on_commit() Callbacks to cover Django 3.2’s captureOnCommitCallbacks() method, and its backport in django-capture- on-commit-callbacks. • Updated section Use setUpTestData to cover Django 3.2’s merging of django- testdata, and link to my “How to convert” post. • Updated section Prevent Output to cover Django 4.0’s --buffer --parallel sup- port, and x error reporting in example code. 10 https://seddonym.me/ 11 https://jeroenjanssens.com/ 12 https://www.seporaitis.net/ 1.5. Changelog 7

📄 Page 13

Speed Up Your Django Tests, Release 2021-07-08 • Updated section Activate Parallelization to better describe the multiprocessing start methods, and re ect the current status of the spawn method ticket. • Updated the parallel testing macOS workaround in a safer way, without setting the environment variable globally. • Moved the Mock Settings section from “Toolbox” to “Targeted Mocking”. • Renamed chapter 12 to “Targeted Mocking” as “targetted” is a misspelling13. • Fixed some errors reported by Matt Layman, Nat Dunn, and Paolo Melchiorre. • Fixed links to pytest documentation after it was reorganized. 2020-05-18 • First release! 13 https://english.stackexchange.com/questions/205815/is-targetted-a-standard-british-english-spelling 8 Chapter 1. Introduction

📄 Page 14

Chapter 2 Toolbox “If all you have is a hammer, everything looks like a nail.” —The Law of the Instrument14 Let’s look at the tools you can use to improve test speed. The following chapters make use of these extensively. 2.1 Test Framework I’m going to cover two test frameworks in this book: • Django’s built-in test framework15 - the one behind python manage.py test. • pytest: helps you write better programs16 - a popular, more Pythonic test runner, that you invoke with pytest. (I’ll also brie y touch on the now-disused framework nose at the end of this section.) 14 https://en.wikipedia.org/wiki/Law_of_the_instrument 15 https://docs.djangoproject.com/en/3.2/topics/testing/ 16 https://pytest.org/en/latest/index.html 9

📄 Page 15

Speed Up Your Django Tests, Release 2021-07-08 Normally Django developers rst learn testing through its built-in framework. Therefore, while popular, pytest sees a bit less use, at least in the Django community. In each section I’ll start with Django’s test framework, then cover how things differ in pytest (if they do). My personal preference is to use pytest. It has a lot of features built-in, it has an extensive plugin ecosystem, and it’s under active development. As we’ll see, many of the things that take extra work with Django’s test framework are included in pytest. However, this book is not intended to convince you to switch. It’s not necessarily feasible to switch framework, especially only to increase your test speed. Although pytest tries to provide drop-in compatibility, it doesn’t always work out - there are always edge cases. Maybe in reading though, you will see more of what pytest is about and try it out. 2.1.1 Django’s Test Framework When we use Django’s test framework, we’re actually using two things: • The standard library’s unittest17 module. It’s based on the jUnit framework from Java, which is why names are in camelCase rather than snake_case. • Django’s test framework. This is comprises extensions to unittest18, such as django.test.TestCase19 and test database setup. As we’ll see, both provide us with a lot of hooks for extending the testing process. How- ever, there’s no plugin system, so if we want to use others’ extensions, we have to copy- paste snippets or use multiple inheritance. 2.1.2 pytest When we use pytest for Django test suites, we need to use: • pytest20 itself • The pytest-django21 plugin. pytest also has many more built-ins and hooks, and there’s a whole ecosystem of plugins that work with it. We’ll touch on many of these throughout the book, but there are so 17 https://docs.python.org/3/library/unittest.html#module-unittest 18 https://docs.python.org/3/library/unittest.html#module-unittest 19 https://docs.djangoproject.com/en/3.2/topics/testing/tools/#django.test.TestCase 20 https://pytest.org/en/latest/index.html 21 https://pytest-django.readthedocs.io/en/latest/index.html 10 Chapter 2. Toolbox

📄 Page 16

Speed Up Your Django Tests, Release 2021-07-08 many, it will still be just a preview. You can nd out more in the excellent documentation for pytest, pytest-django, and the many blog posts and plugins they link to. pytest has built-in support for unittest22 test cases and pytest-django23 reorganizes parts of Django’s test framework to work under pytest. With pytest-django set up, pytest can run existing Django test cases with no modi cation, so converting to pytest can be fairly straightforward. See pytest-django’s Getting Started Guide24. There are four styles we can use for Django tests running under pytest: 1. Django’s TestCase with its self.assert* functions: from django.test import SimpleTestCase class AdditionTests(SimpleTestCase): def test_one(self): self.assertEqual(2 + 2, 4) This style is the same as using Django’s test framework. 2. Django’s TestCase with plain assert statements: from django.test import SimpleTestCase class AdditionTests(SimpleTestCase): def test_one(self): assert 2 + 2 == 4 This style takes advantage of pytest’s assertion rewriting25, which decomposes assert statements on failure to show the components. 3. Plain test functions: def test_one(): assert 2 + 2 == 4 pytest collects module level functions with names starting “test_” as tests. This style makes it easy to start writing tests. 4. Plain test classes: 22 https://docs.python.org/3/library/unittest.html#module-unittest 23 https://pytest-django.readthedocs.io/en/latest/index.html 24 https://pytest-django.readthedocs.io/en/latest/tutorial.html 25 https://pytest.org/en/latest/how-to/assert.html 2.1. Test Framework 11

📄 Page 17

Speed Up Your Django Tests, Release 2021-07-08 class TestAddition: def test_one(self): assert 2 + 2 == 4 pytest nds module-level classes with names starting “Test”, and collects their methods with names starting “test_” as tests. There’s no base class to inherit from. This style also makes it easy to start writing tests, with some grouping. The rst style, using vanilla TestCases does not take advantage of any pytest features. This style is mostly useful as a rst step when converting to pytest. Using the TestCase. assert* functions requires extra learning and recall for developers, and reduces read- ability, compared to the assert statement. So it’s normally better to convert any such such function calls to the assert statement, as in the second style. There’s even a pytest tool called unittest2pytest26 that can help you do the conversion. The third style, using functions, is “the pytest default” as it appears most in the pytest documentation, although the fourth class-based style is just as well supported. Either style allows us to use pytest markers and xtures seamlessly, including those in pytest- django27.. Using classes, as in the fourth style, is a bit more maintainable in the long run, tests for the same thing, such as a view, can be grouped together. That said there’s a downside to using plain test functions or classes. Not every feature from Django’s test framework is available via pytest-django, and some would be espe- cially hard for it to copy, such as Django’s class-level transactions. Going in the other di- rection is possible: whilst a little clunky, we can use pytest xtures in TestCase classes28. Therefore, my preference is to use the second style, using the best of both Django’s test framework with pytest’s features. During this book we’ll be using the rst style. This allows the examples to work on both frameworks, and allows us to use all the features in Django’s testing framework. 2.1.3 Historical: nose nose29 was the previous “better python test runner”, popular before pytest took over. At time of writing it is long abandoned, with the last commit four years ago in 2016. It was popular with Django projects too, via the django-nose30 project. One of its best features was parallel test execution, which it supported before Django did. 26 https://github.com/pytest-dev/unittest2pytest 27 https://pytest-django.readthedocs.io/en/latest/helpers.html 28 https://pytest.org/en/latest/how-to/unittest.html#mixing-fixtures 29 https://nose.readthedocs.io/en/latest/ 30 https://django-nose.readthedocs.io/en/latest/ 12 Chapter 2. Toolbox

📄 Page 18

Speed Up Your Django Tests, Release 2021-07-08 The nose2 project31 was started as a rewrite, but it hasn’t been as successful as its pre- decessor. It’s still going but I don’t know of any maintained Django runner. If you have an old project using nose, it’s probably better at this time to move to ei- ther Django’s test runner or pytest. pytest has some nose compatibility features, see its documentation page How to run tests written for nose32. 2.2 Custom test Management Command Note: This section only applies to the Django test framework. pytest does not use a management command to run the tests. This is the outermost layer of the test process. When you run python manage.py test, Django loads and runs the “test” management command. Normally it nds and uses the built-in command. But if you add a custom management command33 called “test” in one of your project’s apps, it will use that instead. If you subclass and override parts of the built-in command, you can extend its functionality. For example, you could create example/core/management/commands/test.py with this content: import platform from django.core.management.commands.test import Command as BaseCommand class Command(BaseCommand): def handle(self, *test_labels, **options): print(f"Python version: {platform.python_version()}") return super().handle(*test_labels, **options) Thenwhen you run your tests, you’ll get amessage with the current Python version before the tests start: $ python manage.py test Python version: 3.8.1 System check identified no issues (0 silenced). (continues on next page) 31 https://docs.nose2.io/en/latest/ 32 https://pytest.org/en/latest/how-to/nose.html 33 https://docs.djangoproject.com/en/3.2/howto/custom-management-commands/ 2.2. Custom test Management Command 13

📄 Page 19

Speed Up Your Django Tests, Release 2021-07-08 (continued from previous page) ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK Youwon’t need a custom “test” command often. If you read the source34, you’ll see Django doesn’t even do much there. The test runner is the most common extension point, and can even add extra options to the management command. Let’s look at that next. 2.3 Test Runner The test runner is the part of the test framework that handles nding and running your tests. You can customize various parts of this process. 2.3.1 With Django’s Test Framework Django’s test process uses a “test runner” class called DiscoverRunner. This co-ordinates the whole test process, including discovery, setup, and execution. You can extend it by creating a custom subclass and pointing the TEST_RUNNER35 setting at your subclass. If you’re using Django’s test framework, this is the main place to add changes that affect the whole test run. For example, imagine you wanted to ensure all warnings are shown, without needing to pass -W to python each time you run the tests. You could add the below content to a le example/tests/runner.py: import warnings from django.test.runner import DiscoverRunner class ExampleTestRunner(DiscoverRunner): def run_tests(self, *args, **kwargs): # Show all warnings once, especially to show DeprecationWarning # messages which Python ignores by default (continues on next page) 34 https://github.com/django/django/blob/stable/3.0.x/django/core/management/commands/test.py 35 https://docs.djangoproject.com/en/3.2/ref/settings/#std:setting-TEST_RUNNER 14 Chapter 2. Toolbox

📄 Page 20

Speed Up Your Django Tests, Release 2021-07-08 (continued from previous page) warnings.simplefilter("default") return super().run_tests(*args, **kwargs) This overrides the run_tests() method, the core method for actually executing test classes and methods. You also need to con gure your runner to be used in your set- tings: TEST_RUNNER = "example.tests.runner.ExampleTestRunner" You can extend many different points of the testing process using similar method over- rides. DiscoverRunner also has some attributes that refer to unittest36 classes that the test runner uses. You can replace these attributes with pointers to your own subclasses to customize some deeper details of the test process. You won’t need these so often, so we’ll cover these when we need to. Django’s methods and attributes are documented in the Django test runner documenta- tion37. 2.3.2 With pytest-django If you are using pytest-django, the TEST_RUNNER setting is ignored38. But pytest offers many granular hooks which you can use to achieve all the same things. You’ll normally make global changes in a conftest.py le39 in the root of your project. For example to add a warning lter we’d need to just edit example/conftest.py to con- tain: import warnings # Show all warnings once, especially to show DeprecationWarning # messages which Python ignores by default warnings.simplefilter("default") 36 https://docs.python.org/3/library/unittest.html#module-unittest 37 https://docs.djangoproject.com/en/3.2/topics/testing/advanced/#topics-testing-test-runner 38 https://github.com/pytest-dev/pytest-django/issues/681 39 https://pytest.org/en/latest/how-to/writing_plugins.html#local-conftest-plugins 2.3. Test Runner 15