The Python programming language has unique strengths and charms that can be hard to grasp. Many programmers familiar with other languages often approach Python from a limited mindset instead of embracing its full expressivity. Some programmers go too far in the other direction, overusing Python features that can cause big problems later.
This second edition book (to be published by Pearson Addison-Wesley in mid-November 2019) provides insight into the Pythonic way of writing programs: the best way to use Python. It builds on a fundamental understanding of the language that I assume you already have. Novice programmers will learn the best practices of Python’s capabilities. Experienced programmers will learn how to embrace the strangeness of a new tool with confidence.
My goal is to prepare you to make a big impact with Python.
Buy the Book on Amazon Buy the DRM-free eBook
Visit the GitHub project to see all of the code snippets from the book in one place. Run and modify the example code yourself to confirm your understanding. You can also report any errors you’ve found.
For future updates about the book, related videos, translations, conference presentations, and more, choose one of these ways to stay in touch:
Each chapter in Effective Python: Second Edition contains a broad but related set of items. Feel free to jump between all 90 items and follow your interest. Each item contains concise and specific guidance explaining how you can write Python programs more effectively. Items include advice on what to do, what to avoid, how to strike the right balance, and why this is the best choice. Items reference each other to make it easier to fill in the gaps as you read.
This second edition is focused exclusively on Python 3, up to and including version 3.8. It contains 30+ completely new items with additional best practices. Most of the original items from the first edition have been revised and included, but many have undergone substantial updates. For some items my advice has completely changed between the two editions of the book due best practices evolving as Python has matured.
If, for whatever reason, you’re still primarily using Python 2, despite its end-of-life on January 1st, 2020, the previous edition of the book (and its translations) may be more useful to you.
The Python community has come to use the adjective Pythonic to describe code that follows a particular style. The idioms of Python have emerged over time through experience using the language and working with others. This chapter covers the best way to do the most common things in Python.
bytes
and str
str.format
enumerate
Over rangezip
to Process Iterators in Parallelelse
Blocks After for
and while
LoopsIn Python, the most common way to organize information is in a sequence of values stored in a list
. A list
‘s natural complement is the dict
that stores lookup keys mapped to corresponding values. This chapter covers how to build programs with these versatile building blocks.
key
Parameterdict
Insertion Orderingget
Over in
and KeyError
to Handle Missing Dictionary Keysdefaultdict
Over setdefault
to Handle Missing Items in Internal State__missing__
Functions in Python have a variety of extra features that make a programmer’s life easier. Some are similar to capabilities in other programming languages, but many are unique to Python. This chapter covers how to use functions to clarify intention, promote reuse, and reduce bugs.
None
None
and Docstrings to Specify Dynamic Default Argumentsfunctools.wraps
Python has special syntax for quickly iterating through lists, dictionaries, and sets to generate derivative data structures. It also allows for a stream of iterable values to be incrementally returned by a function. This chapter covers how these features can provide better performance, reduced memory usage, and improved readability.
map
and filter
yield from
send
throw
itertools
for Working with Iterators and GeneratorsPython is an object-oriented language. Getting things done in Python often requires writing new classes and defining how they interact through their interfaces and hierarchies. This chapter covers how to use classes to express your intended behaviors with objects.
@classmethod
Polymorphism to Construct Objects Genericallysuper
collections.abc
for Custom Container TypesMetaclasses and dynamic attributes are powerful Python features. However, they also enable you to implement extremely bizarre and unexpected behaviors. This chapter covers the common idioms for using these mechanisms to ensure that you follow the rule of least surprise.
@property
Instead of Refactoring Attributes@property
Methods__getattr__
, __getattribute__
, and __setattr__
for Lazy Attributes__init_subclass__
__init_subclass__
__set_name__
Python makes it easy to write concurrent programs that do many different things seemingly at the same time. Python can also be used to do parallel work through system calls, subprocesses, and C extensions. This chapter covers how to best utilize Python in these subtly different situations.
subprocess
to Manage Child ProcessesLock
to Prevent Data Races in ThreadsQueue
to Coordinate Work Between ThreadsThread
Instances for On-demand Fan-outQueue
for Concurrency Requires RefactoringThreadPoolExecutor
When Threads Are Necessary for Concurrencyasyncio
asyncio
asyncio
Event Loop to Maximize Responsivenessconcurrent.futures
for True ParallelismPython has built-in features and modules that aid in hardening your programs so they are dependable. Python also includes tools to help you achieve higher performance with minimal effort. This chapter covers how to use Python to optimize your programs to maximize their reliability and efficiency in production.
try
/except
/else
/finally
contextlib
and with
Statements for Reusable try
/finally
Behaviordatetime
Instead of time
for Local Clockspickle
Reliable with copyreg
decimal
When Precision Is Paramountdeque
for Producer–Consumer Queuesbisect
heapq
for Priority Queuesmemoryview
and bytearray
for Zero-Copy Interactions with bytesYou should always test your code, regardless of what language it’s written in. However, Python’s dynamic features can increase the risk of runtime errors in unique ways. Luckily, they also make it easier to write tests and diagnose malfunctioning programs. This chapter covers Python’s built-in tools for testing and debugging.
repr
Strings for Debugging OutputTestCase
SubclassessetUp
, tearDown
, setUpModule
, and tearDownModule
pdb
tracemalloc
to Understand Memory Usage and LeaksCollaborating on Python programs requires you to be deliberate about how you write your code. Even if you’re working alone, you’ll want to understand how to use modules written by others. This chapter covers the standard tools and best practices that enable people to work together on Python programs.
Exception
to Insulate Callers from APIswarnings
to Refactor and Migrate Usagetyping
to Obviate BugsAn assignment expression—also known as the walrus operator—is a new syntax introduced in Python 3.8 to solve a long-standing problem with the language that can cause code duplication. Whereas normal assignment statements are written a = b
and pronounced “a equals b”, these assignments are written a := b
and pronounced “a walrus b” (because :=
looks like a pair of eyeballs and tusks).
Continue reading »
Although metaclasses allow you to customize class creation in multiple ways (see Item 48: “Validate Subclasses with __init_subclass__
” and Item 49: “Register Class Existence with __init_subclass__
”), they still fall short of handling every situation that may arise.
Continue reading »
You can immediately read the second edition of Effective Python as a DRM-free eBook, on Kindle, on Google Play, and on O’Reilly Online Learning.
memoryview
and bytearray
for Zero-Copy Interactions with bytes
Though Python isn’t able to parallelize CPU-bound computation without extra effort (see Item 64: “Consider concurrent.futures
for True Parallelism”), it is able to support high-throughput, parallel I/O in a variety of ways (see Item 53: “Use Threads for Blocking I/O, Avoid for Parallelism” and Item 60: “Achieve Highly Concurrent I/O with Coroutines” for details). That said, it’s surprisingly easy to use these I/O tools the wrong way and reach the conclusion that the language is too slow for even I/O-bound workloads.
Continue reading »
Effective Python: Second Edition is now available for preorder. Follow this link to buy your copy in advance. It will ship in mid-November (2019) once the book has finished printing and is stocked in the warehouse. Digital editions will become available when the physical book ships or sooner.