The Python programming language has unique strengths and charms that can be hard to grasp. Many programmers familiar with other languages often approach Python from a limited mindset instead of embracing its full expressivity. Some programmers go too far in the other direction, overusing Python features that can cause big problems later.
This second edition book (to be published by Pearson Addison-Wesley in mid-November 2019) provides insight into the Pythonic way of writing programs: the best way to use Python. It builds on a fundamental understanding of the language that I assume you already have. Novice programmers will learn the best practices of Python’s capabilities. Experienced programmers will learn how to embrace the strangeness of a new tool with confidence.
My goal is to prepare you to make a big impact with Python.
Visit the GitHub project to see all of the code snippets from the book in one place. Run and modify the example code yourself to confirm your understanding. You can also report any errors you’ve found.
For future updates about the book, related videos, translations, conference presentations, and more, choose one of these ways to stay in touch:
Each chapter in Effective Python: Second Edition contains a broad but related set of items. Feel free to jump between all 90 items and follow your interest. Each item contains concise and specific guidance explaining how you can write Python programs more effectively. Items include advice on what to do, what to avoid, how to strike the right balance, and why this is the best choice. Items reference each other to make it easier to fill in the gaps as you read.
This second edition is focused exclusively on Python 3, up to and including version 3.8. It contains 30+ completely new items with additional best practices. Most of the original items from the first edition have been revised and included, but many have undergone substantial updates. For some items my advice has completely changed between the two editions of the book due best practices evolving as Python has matured.
If, for whatever reason, you’re still primarily using Python 2, despite its end-of-life on January 1st, 2020, the previous edition of the book (and its translations) may be more useful to you.
The Python community has come to use the adjective Pythonic to describe code that follows a particular style. The idioms of Python have emerged over time through experience using the language and working with others. This chapter covers the best way to do the most common things in Python.
zipto Process Iterators in Parallel
In Python, the most common way to organize information is in a sequence of values stored in a
list‘s natural complement is the
dict that stores lookup keys mapped to corresponding values. This chapter covers how to build programs with these versatile building blocks.
KeyErrorto Handle Missing Dictionary Keys
setdefaultto Handle Missing Items in Internal State
Functions in Python have a variety of extra features that make a programmer’s life easier. Some are similar to capabilities in other programming languages, but many are unique to Python. This chapter covers how to use functions to clarify intention, promote reuse, and reduce bugs.
Noneand Docstrings to Specify Dynamic Default Arguments
Python has special syntax for quickly iterating through lists, dictionaries, and sets to generate derivative data structures. It also allows for a stream of iterable values to be incrementally returned by a function. This chapter covers how these features can provide better performance, reduced memory usage, and improved readability.
itertoolsfor Working with Iterators and Generators
Python is an object-oriented language. Getting things done in Python often requires writing new classes and defining how they interact through their interfaces and hierarchies. This chapter covers how to use classes to express your intended behaviors with objects.
@classmethodPolymorphism to Construct Objects Generically
collections.abcfor Custom Container Types
Metaclasses and dynamic attributes are powerful Python features. However, they also enable you to implement extremely bizarre and unexpected behaviors. This chapter covers the common idioms for using these mechanisms to ensure that you follow the rule of least surprise.
@propertyInstead of Refactoring Attributes
__setattr__for Lazy Attributes
Python makes it easy to write concurrent programs that do many different things seemingly at the same time. Python can also be used to do parallel work through system calls, subprocesses, and C extensions. This chapter covers how to best utilize Python in these subtly different situations.
subprocessto Manage Child Processes
Lockto Prevent Data Races in Threads
Queueto Coordinate Work Between Threads
ThreadInstances for On-demand Fan-out
Queuefor Concurrency Requires Refactoring
ThreadPoolExecutorWhen Threads Are Necessary for Concurrency
asyncioEvent Loop to Maximize Responsiveness
concurrent.futuresfor True Parallelism
Python has built-in features and modules that aid in hardening your programs so they are dependable. Python also includes tools to help you achieve higher performance with minimal effort. This chapter covers how to use Python to optimize your programs to maximize their reliability and efficiency in production.
withStatements for Reusable
timefor Local Clocks
decimalWhen Precision Is Paramount
dequefor Producer–Consumer Queues
heapqfor Priority Queues
bytearrayfor Zero-Copy Interactions with bytes
You should always test your code, regardless of what language it’s written in. However, Python’s dynamic features can increase the risk of runtime errors in unique ways. Luckily, they also make it easier to write tests and diagnose malfunctioning programs. This chapter covers Python’s built-in tools for testing and debugging.
reprStrings for Debugging Output
tracemallocto Understand Memory Usage and Leaks
Collaborating on Python programs requires you to be deliberate about how you write your code. Even if you’re working alone, you’ll want to understand how to use modules written by others. This chapter covers the standard tools and best practices that enable people to work together on Python programs.
Exceptionto Insulate Callers from APIs
warningsto Refactor and Migrate Usage
typingto Obviate Bugs
An assignment expression—also known as the walrus operator—is a new syntax introduced in Python 3.8 to solve a long-standing problem with the language that can cause code duplication. Whereas normal assignment statements are written
a = b and pronounced “a equals b”, these assignments are written
a := b and pronounced “a walrus b” (because
:= looks like a pair of eyeballs and tusks).
Continue reading »
Although metaclasses allow you to customize class creation in multiple ways (see Item 48: “Validate Subclasses with
__init_subclass__” and Item 49: “Register Class Existence with
__init_subclass__”), they still fall short of handling every situation that may arise.
Continue reading »
Though Python isn’t able to parallelize CPU-bound computation without extra effort (see Item 64: “Consider
concurrent.futures for True Parallelism”), it is able to support high-throughput, parallel I/O in a variety of ways (see Item 53: “Use Threads for Blocking I/O, Avoid for Parallelism” and Item 60: “Achieve Highly Concurrent I/O with Coroutines” for details). That said, it’s surprisingly easy to use these I/O tools the wrong way and reach the conclusion that the language is too slow for even I/O-bound workloads.
Continue reading »