Python Packaging User Guide

Python Packaging Authority

Apr 23, 2019

 Contents

1 An Overview of Packaging for Python 1

1.1 Thinking about deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Packaging Python libraries and tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Packaging Python applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 What about. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Wrap up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Tutorials 9
2.1 Installing Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Managing Application Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3 Packaging Python Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Creating Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3 Guides 25
3.1 Tool recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Installing packages using pip and virtual environments . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 Installing stand alone command line tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Installing pip/setuptools/wheel with Linux Package Managers . . . . . . . . . . . . . . . . . . . . . 33
3.5 Installing scientific packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.6 Multi-version installs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.7 Packaging and distributing projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.8 Single-sourcing the package version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.9 Supporting multiple Python versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.10 Dropping support for older Python versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.11 Packaging binary extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.12 Supporting Windows using Appveyor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.13 Packaging namespace packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.14 Creating and discovering plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.15 Analyzing PyPI package downloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.16 Package index mirrors and caches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.17 Hosting your own simple repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.18 Migrating to [Link] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.19 Using TestPyPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.20 Making a PyPI-friendly README . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

4 Discussions 83
4.1 Deploying Python applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

i
 4.2 pip vs easy_install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3 install_requires vs requirements files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4 Wheel vs Egg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5 PyPA specifications 89
5.1 Package Distribution Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.2 Package Index Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

6 Project Summaries 103

6.1 PyPA Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2 Non-PyPA Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.3 Standard Library Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

7 Glossary 109

8 How to Get Support 113

9 Contribute to this guide 115

9.1 Documentation types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
9.2 Building the guide locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
9.3 Where the guide is deployed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
9.4 Style guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

10 News 119
10.1 September 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.2 August 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.3 July 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.4 June 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.5 May 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.6 April 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.7 March 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.8 February 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.9 January 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.10 December 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.11 November 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.12 October 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.13 September 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.14 August 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.15 July 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.16 June 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.17 May 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.18 April 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.19 March 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
10.20 February 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

11 Get started 125

12 Learn more 127

ii
 CHAPTER 1

An Overview of Packaging for Python

As a general-purpose programming language, Python is designed to be used in many ways. You can build web sites
or industrial robots or a game for your friends to play, and much more, all using the same core technology.
Python’s flexibility is why the first step in every Python project must be to think about the project’s audience and the
corresponding environment where the project will run. It might seem strange to think about packaging before writing
code, but this process does wonders for avoiding future headaches.
This overview provides a general-purpose decision tree for reasoning about Python’s plethora of packaging options.
Read on to choose the best technology for your next project.

Contents

• Thinking about deployment

• Packaging Python libraries and tools
– Python modules
– Python source distributions
– Python binary distributions
• Packaging Python applications
– Depending on a framework

* Service platforms
* Web browsers and mobile applications
– Depending on a pre-installed Python
– Depending on a separate software distribution ecosystem
– Bringing your own Python executable
– Bringing your own userspace

1
Python Packaging User Guide

– Bringing your own kernel

– Bringing your own hardware
• What about. . .
– Operating system packages
– virtualenv
– Security
• Wrap up

1.1 Thinking about deployment

Packages exist to be installed (or deployed), so before you package anything, you’ll want to have some answers to the
deployment questions below:
• Who are your software’s users? Will your software be installed by other developers doing software development,
operations people in a datacenter, or a less software-savvy group?
• Is your software intended to run on servers, desktops, mobile clients (phones, tablets, etc.), or embedded in
dedicated devices?
• Is your software installed individually, or in large deployment batches?
Packaging is all about target environment and deployment experience. There are many answers to the questions above
and each combination of circumstances has its own solutions. With this information, the following overview will guide
you to the packaging technologies best suited to your project.

1.2 Packaging Python libraries and tools

You may have heard about PyPI, [Link], and wheel files. These are just a few of the tools Python’s ecosystem
provides for distributing Python code to developers, which you can read about in Packaging and distributing projects.
The following approaches to packaging are meant for libraries and tools used by technical audience in a development
setting. If you’re looking for ways to package Python for a non-technical audience and/or a production setting, skip
ahead to Packaging Python applications.

1.2.1 Python modules

A Python file, provided it only relies on the standard library, can be redistributed and reused. You will also need to
ensure it’s written for the right version of Python, and only relies on the standard library.
This is great for sharing simple scripts and snippets between people who both have compatible Python versions (such
as via email, StackOverflow, or GitHub gists). There are even some entire Python libraries that offer this as an option,
such as [Link] and boltons.
However, this pattern won’t scale for projects that consist of multiple files, need additional libraries, or need a specific
version of Python, hence the options below.

2 Chapter 1. An Overview of Packaging for Python

 Python Packaging User Guide

1.2.2 Python source distributions

If your code consists of multiple Python files, it’s usually organized into a directory structure. Any directory containing
Python files can comprise an import package.
Because packages consist of multiple files, they are harder to distribute. Most protocols support transferring only one
file at a time (when was the last time you clicked a link and it downloaded multiple files?). It’s easier to get incomplete
transfers, and harder to guarantee code integrity at the destination.
So long as your code contains nothing but pure Python code, and you know your deployment environment supports
your version of Python, then you can use Python’s native packaging tools to create a source distribution package, or
sdist for short.
Python’s sdists are compressed archives (.[Link] files) containing one or more packages or modules. If your code
is pure-Python, and you only depend on other Python packages, you can go here to learn more.
If you rely on any non-Python code, or non-Python packages (such as libxml2 in the case of lxml, or BLAS libraries
in the case of numpy), you will need to use the format detailed in the next section, which also has many advantages
for pure-Python libraries.

Note: Python and PyPI support multiple distributions providing different implementations of the same package. For
instance the unmaintained-but-seminal PIL distribution provides the PIL package, and so does Pillow, an actively-
maintained fork of PIL!
This Python packaging superpower makes it possible for Pillow to be a drop-in replacement for PIL, just by changing
your project’s install_requires or [Link].

1.2.3 Python binary distributions

So much of Python’s practical power comes from its ability to integrate with the software ecosystem, in particular
libraries written in C, C++, Fortran, Rust, and other languages.
Not all developers have the right tools or experiences to build these components written in these compiled languages,
so Python created the wheel, a package format designed to ship libraries with compiled artifacts. In fact, Python’s
package installer, pip, always prefers wheels because installation is always faster, so even pure-Python packages
work better with wheels.
Binary distributions are best when they come with source distributions to match. Even if you don’t upload wheels of
your code for every operating system, by uploading the sdist, you’re enabling users of other platforms to still build it
for themselves. Default to publishing both sdist and wheel archives together, unless you’re creating artifacts for a very
specific use case where you know the recipient only needs one or the other.
Python and PyPI make it easy to upload both wheels and sdists together. Just follow the Packaging Python Projects
tutorial.

1.3 Packaging Python applications

So far we’ve only discussed Python’s native distribution tools. Based on our introduction, you would be correct to
infer these built-in approaches only target environments which have Python, and an audience who knows how to install
Python packages.
With the variety of operating systems, configurations, and people out there, this assumption is only safe when targeting
a developer audience.

1.3. Packaging Python applications 3

Python Packaging User Guide

Fig. 1: Python’s recommended built-in library and tool packaging technologies. Excerpted from The Packaging
Gradient (2017).

Python’s native packaging is mostly built for distributing reusable code, called libraries, between developers. You can
piggyback tools, or basic applications for developers, on top of Python’s library packaging, using technologies like
setuptools entry_points.
Libraries are building blocks, not complete applications. For distributing applications, there’s a whole new world of
technologies out there.
The next few sections organize these application packaging options according to their dependencies on the target
environment, so you can choose the right one for your project.

1.3.1 Depending on a framework

Some types of Python applications, like web site backends and other network services, are common enough that they
have frameworks to enable their development and packaging. Other types of applications, like dynamic web frontends
and mobile clients, are complex enough to target that a framework becomes more than a convenience.
In all these cases, it makes sense to work backwards, from the framework’s packaging and deployment story. Some
frameworks include a deployment system which wraps the technologies outlined in the rest of the guide. In these cases,
you’ll want to defer to your framework’s packaging guide for the easiest and most reliable production experience.
If you ever wonder how these platforms and frameworks work under the hood, you can always read the sections
beyond.

Service platforms

If you’re developing for a “Platform-as-a-Service” or “PaaS” like Heroku or Google App Engine, you are going to
want to follow their respective packaging guides.
• Heroku
• Google App Engine
• PythonAnywhere
• OpenShift
• “Serverless” frameworks like Zappa
In all these setups, the platform takes care of packaging and deployment, as long as you follow their patterns. Most
software does not fit one of these templates, hence the existence of all the other options below.

4 Chapter 1. An Overview of Packaging for Python

 Python Packaging User Guide

If you’re developing software that will be deployed to machines you own, users’ personal computers, or any other
arrangement, read on.

Web browsers and mobile applications

Python’s steady advances are leading it into new spaces. These days you can write a mobile app or web application
frontend in Python. While the language may be familiar, the packaging and deployment practices are brand new.
If you’re planning on releasing to these new frontiers, you’ll want to check out the following frameworks, and refer to
their packaging guides:
• Kivy
• Beeware
• Brython
• Flexx
If you are not interested in using a framework or platform, or just wonder about some of the technologies and tech-
niques utilized by the frameworks above, continue reading below.

1.3.2 Depending on a pre-installed Python

Pick an arbitrary computer, and depending on the context, there’s a very good chance Python is already installed.
Included by default in most Linux and Mac operating systems for many years now, you can reasonably depend on
Python preexisting in your data centers or on the personal machines of developers and data scientists.
Technologies which support this model:
• PEX (Python EXecutable)
• zipapp (does not help manage dependencies, requires Python 3.5+)
• shiv (requires Python 3)

Note: Of all the approaches here, depending on a pre-installed Python relies the most on the target environment. Of
course, this also makes for the smallest package, as small as single-digit megabytes, or even kilobytes.
In general, decreasing the dependency on the target system increases the size of our package, so the solutions here are
roughly arranged by increasing size of output.

1.3.3 Depending on a separate software distribution ecosystem

For a long time many operating systems, including Mac and Windows, lacked built-in package management. Only
recently did these OSes gain so-called “app stores”, but even those focus on consumer applications and offer little for
developers.
Developers long sought remedies, and in this struggle, emerged with their own package management solutions, such as
Homebrew. The most relevant alternative for Python developers is a package ecosystem called Anaconda. Anaconda
is built around Python and is increasingly common in academic, analytical, and other data-oriented environments, even
making its way into server-oriented environments.
Instructions on building and publishing for the Anaconda ecosystem:
• Building libraries and applications with conda

1.3. Packaging Python applications 5

Python Packaging User Guide

• Transitioning a native Python package to Anaconda

A similar model involves installing an alternative Python distribution, but does not support arbitrary operating system-
level packages:
• Enthought Canopy
• ActiveState ActivePython
• WinPython

1.3.4 Bringing your own Python executable

Computing as we know it is defined by the ability to execute programs. Every operating system natively supports one
or more formats of program they can natively execute.
There are many techniques and technologies which turn your Python program into one of these formats, most of which
involve embedding the Python interpreter and any other dependencies into a single executable file.
This approach, called freezing, offers wide compatiblity and seamless user experience, though often requires multiple
technologies, and a good amount of effort.
A selection of Python freezers:
• pyInstaller - Cross-platform
• cx_Freeze - Cross-platform
• constructor - For command-line installers
• py2exe - Windows only
• py2app - Mac only
• bbFreeze - Windows, Linux, Python 2 only
• osnap - Windows and Mac
• pynsist - Windows only
Most of the above imply single-user deployments. For multi-component server applications, see Chef Omnibus.

1.3.5 Bringing your own userspace

An increasing number of operating systems – including Linux, Mac OS, and Windows – can be set up to run appli-
cations packaged as lightweight images, using a relatively modern arrangement often referred to as operating-system-
level virtualization, or containerization.
These techniques are mostly Python agnostic, because they package whole OS filesystems, not just Python or Python
packages.
Adoption is most extensive among Linux servers, where the technology originated and where the technologies below
work best:
• AppImage
• Docker
• Flatpak
• Snapcraft

6 Chapter 1. An Overview of Packaging for Python

 Python Packaging User Guide

1.3.6 Bringing your own kernel

Most operating systems support some form of classical virtualization, running applications packaged as images con-
taining a full operating system of their own. Running these virtual machines, or VMs, is a mature approach, widespread
in data center environments.
These techniques are mostly reserved for larger scale deployments in data centers, though certain complex applications
can benefit from this packaging. Technologies are Python agnostic, and include:
• Vagrant
• VHD, AMI, and other formats
• OpenStack - A cloud management system in Python, with extensive VM support

1.3.7 Bringing your own hardware

The most all-encompassing way to ship your software would be to ship it already-installed on some hardware. This
way, your software’s user would require only electricity.
Whereas the virtual machines described above are primarily reserved for the tech-savvy, you can find hardware appli-
ances being used by everyone from the most advanced data centers to the youngest children.
Embed your code on an Adafruit, MicroPython, or more-powerful hardware running Python, then ship it to the data-
center or your users’ homes. They plug and play, and you can call it a day.

Fig. 2: The simplified gamut of technologies used to package Python applications.

1.4 What about. . .

The sections above can only summarize so much, and you might be wondering about some of the more conspicuous
gaps.

1.4.1 Operating system packages

As mentioned in Depending on a separate software distribution ecosystem above, some operating systems have pack-
age managers of their own. If you’re very sure of the operating system you’re targeting, you can depend directly on a
format like deb (for Debian, Ubuntu, etc.) or RPM (for Red Hat, Fedora, etc.), and use that built-in package manager
to take care of installation, and even deployment. You can even use FPM to generate both deb and RPMs from the
same source.

1.4. What about. . . 7

Python Packaging User Guide

In most deployment pipelines, the OS package manager is just one piece of the puzzle.

1.4.2 virtualenv

Virtualenvs have been an indispensible tool for multiple generations of Python developer, but are slowly fading from
view, as they are being wrapped by higher-level tools. With packaging in particular, virtualenvs are used as a primitive
in the dh-virtualenv tool and osnap, both of which wrap virtualenvs in a self-contained way.
For production deployments, do not rely on running pip install from the Internet into a virtualenv, as one might
do in a development environment. The overview above is full of much better solutions.

1.4.3 Security

The further down the gradient you come, the harder it gets to update components of your package. Everything is more
tightly bound together.
For example, if a kernel security issue emerges, and you’re deploying containers, the host system’s kernel can be
updated without requiring a new build on behalf of the application. If you deploy VM images, you’ll need a new
build. Whether or not this dynamic makes one option more secure is still a bit of an old debate, going back to the
still-unsettled matter of static versus dynamic linking.

1.5 Wrap up

Packaging in Python has a bit of a reputation for being a bumpy ride. This impression is mostly a byproduct of Python’s
versatility. Once you understand the natural boundaries between each packaging solution, you begin to realize that
the varied landscape is a small price Python programmers pay for using one of the most balanced, flexible language
available.

8 Chapter 1. An Overview of Packaging for Python

 CHAPTER 2

Tutorials

Tutorials are opinionated step-by-step guides to help you get familiar with packaging concepts. For more detailed
information on specific packaging topics, see Guides.

2.1 Installing Packages

This section covers the basics of how to install Python packages.

It’s important to note that the term “package” in this context is being used as a synonym for a distribution (i.e. a
bundle of software to be installed), not to refer to the kind of package that you import in your Python source code (i.e.
a container of modules). It is common in the Python community to refer to a distribution using the term “package”.
Using the term “distribution” is often not preferred, because it can easily be confused with a Linux distribution, or
another larger software distribution like Python itself.

Contents

• Requirements for Installing Packages

– Ensure you can run Python from the command line
– Ensure you can run pip from the command line
– Ensure pip, setuptools, and wheel are up to date
– Optionally, create a virtual environment
• Creating Virtual Environments
• Use pip for Installing
• Installing from PyPI
• Source Distributions vs Wheels
• Upgrading packages

9
Python Packaging User Guide

• Installing to the User Site

• Requirements files
• Installing from VCS
• Installing from other Indexes
• Installing from a local src tree
• Installing from local archives
• Installing from other sources
• Installing Prereleases
• Installing Setuptools “Extras”

2.1.1 Requirements for Installing Packages

This section describes the steps to follow before installing other Python packages.

Ensure you can run Python from the command line

Before you go any further, make sure you have Python and that the expected version is available from your command
line. You can check this by running:

python --version

You should get some output like Python 3.6.3. If you do not have Python, please install the latest 3.x version
from [Link] or refer to the Installing Python section of the Hitchhiker’s Guide to Python.

Note: If you’re a newcomer and you get an error like this:

>>> python --version

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined

It’s because this command and other suggested commands in this tutorial are intended to be run in a shell (also called
a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating
system’s shell and interacting with Python.

Note: If you’re using an enhanced shell like IPython or the Jupyter notebook, you can run system commands like
those in this tutorial by prefacing them with a ! character:

In [1]: import sys

!{[Link]} --version
Python 3.6.3

It’s recommended to write {[Link]} rather than plain python in order to ensure that commands are
run in the Python installation matching the currently running notebook (which may not be the same Python installation
that the python command refers to).

10 Chapter 2. Tutorials
 Python Packaging User Guide

Note: Due to the way most Linux distributions are handling the Python 3 migration, Linux users using the sys-
tem Python without creating a virtual environment first should replace the python command in this tutorial with
python3 and the pip command with pip3 --user. Do not run any of the commands in this tutorial with sudo:
if you get a permissions error, come back to the section on creating virtual environments, set one up, and then continue
with the tutorial as written.

Ensure you can run pip from the command line

Additionally, you’ll need to make sure you have pip available. You can check this by running:

pip --version

If you installed Python from source, with an installer from [Link], or via Homebrew you should already have pip.
If you’re on Linux and installed using your OS package manager, you may have to install pip separately, see Installing
pip/setuptools/wheel with Linux Package Managers.
If pip isn’t already installed, then first try to bootstrap it from the standard library:

python -m ensurepip --default-pip

If that still doesn’t allow you to run pip:

• Securely Download get-pip.py1
• Run python [Link].2 This will install or upgrade pip. Additionally, it will install setuptools and
wheel if they’re not installed already.

Warning: Be cautious if you’re using a Python install that’s managed by your operating system or another
package manager. [Link] does not coordinate with those tools, and may leave your system in an inconsis-
tent state. You can use python [Link] --prefix=/usr/local/ to install in /usr/local
which is designed for locally-installed software.

Ensure pip, setuptools, and wheel are up to date

While pip alone is sufficient to install from pre-built binary archives, up to date copies of the setuptools and
wheel projects are useful to ensure you can also install from source archives:

python -m pip install --upgrade pip setuptools wheel

Optionally, create a virtual environment

See section below for details, but here’s the basic venv3 command to use on a typical Linux system:

python3 -m venv tutorial_env

source tutorial_env/bin/activate

1 “Secure” in this context means using a modern browser or a tool like curl that verifies SSL certificates when downloading from https URLs.
2 Depending on your platform, this may require root or Administrator access. pip is currently considering changing this by making user installs
the default behavior.
3 Beginning with Python 3.4, venv (a stdlib alternative to virtualenv) will create virtualenv environments with pip pre-installed, thereby

making it an equal alternative to virtualenv.

2.1. Installing Packages 11

Python Packaging User Guide

This will create a new virtual environment in the tutorial_env subdirectory, and configure the current shell to use
it as the default python environment.

2.1.2 Creating Virtual Environments

Python “Virtual Environments” allow Python packages to be installed in an isolated location for a particular applica-
tion, rather than being installed globally. If you are looking to safely install global command line tools, see Installing
stand alone command line tools.
Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How
can you use both these applications? If you install everything into /usr/lib/python3.6/site-packages (or whatever your
platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that
shouldn’t be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in
its libraries or the versions of those libraries can break the application.
Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.
In all these cases, virtual environments can help you. They have their own installation directories and they don’t share
libraries with other virtual environments.
Currently, there are two common tools for creating Python virtual environments:
• venv is available by default in Python 3.3 and later, and installs pip and setuptools into created virtual environ-
ments in Python 3.4 and later.
• virtualenv needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and pip, setuptools and
wheel are always installed into created virtual environments by default (regardless of Python version).
The basic usage is like so:
Using venv:

python3 -m venv <DIR>

source <DIR>/bin/activate

Using virtualenv:

virtualenv <DIR>
source <DIR>/bin/activate

For more information, see the venv docs or the virtualenv docs.
In both of the above cases, Windows users should _not_ use the source command, but should rather run the activate
script directly from the command shell. The use of source under Unix shells ensures that the virtual environment’s
variables are set within the current shell, and not in a subprocess (which then disappears, having no useful effect).
Managing multiple virtual environments directly can become tedious, so the dependency management tutorial in-
troduces a higher level tool, Pipenv, that automatically manages a separate virtual environment for each project and
application that you work on.

2.1.3 Use pip for Installing

pip is the recommended installer. Below, we’ll cover the most common usage scenarios. For more detail, see the pip
docs, which includes a complete Reference Guide.

12 Chapter 2. Tutorials
 Python Packaging User Guide

2.1.4 Installing from PyPI

The most common usage of pip is to install from the Python Package Index using a requirement specifier. Generally
speaking, a requirement specifier is composed of a project name followed by an optional version specifier. PEP 440
contains a full specification of the currently supported specifiers. Below are some examples.
To install the latest version of “SomeProject”:
pip install "SomeProject"

To install a specific version:

pip install "SomeProject==1.4"

To install greater than or equal to one version and less than another:
pip install "SomeProject>=1,<2"

To install a version that’s “compatible” with a certain version:4

pip install "SomeProject~=1.4.2"

In this case, this means to install any version “==1.4.*” version that’s also “>=1.4.2”.

2.1.5 Source Distributions vs Wheels

pip can install from either Source Distributions (sdist) or Wheels, but if both are present on PyPI, pip will prefer a
compatible wheel.
Wheels are a pre-built distribution format that provides faster installation compared to Source Distributions (sdist),
especially when a project contains compiled extensions.
If pip does not find a wheel to install, it will locally build a wheel and cache it for future installs, instead of rebuilding
the source distribution in the future.

2.1.6 Upgrading packages

Upgrade an already installed SomeProject to the latest from PyPI.

pip install --upgrade SomeProject

2.1.7 Installing to the User Site

To install packages that are isolated to the current user, use the --user flag:
pip install --user SomeProject

For more information see the User Installs section from the pip docs.
Note that the --user flag has no effect when inside a virtual environment - all installation commands will affect the
virtual environment.
If SomeProject defines any command-line scripts or console entry points, --user will cause them to be installed
inside the user base’s binary directory, which may or may not already be present in your shell’s PATH. (Starting in
4 The compatible release specifier was accepted in PEP 440 and support was released in setuptools v8.0 and pip v6.0

2.1. Installing Packages 13

Python Packaging User Guide

version 10, pip displays a warning when installing any scripts to a directory outside PATH.) If the scripts are not
available in your shell after installation, you’ll need to add the directory to your PATH:
• On Linux and macOS you can find the user base binary directory by running python -m site
--user-base and adding bin to the end. For example, this will typically print ~/.local (with ~ ex-
panded to the absolute path to your home directory) so you’ll need to add ~/.local/bin to your PATH. You
can set your PATH permanently by modifying ~/.profile.
• On Windows you can find the user base binary directory by running py -m site
--user-site and replacing site-packages with Scripts. For example, this could return
C:\Users\Username\AppData\Roaming\Python36\site-packages so you would need to
set your PATH to include C:\Users\Username\AppData\Roaming\Python36\Scripts. You can
set your user PATH permanently in the Control Panel. You may need to log out for the PATH changes to take
effect.

2.1.8 Requirements files

Install a list of requirements specified in a Requirements File.

pip install -r [Link]

2.1.9 Installing from VCS

Install a project from VCS in “editable” mode. For a full breakdown of the syntax, see pip’s section on VCS Support.

pip install -e git+[Link] # from git

pip install -e hg+[Link] # from
˓→mercurial

pip install -e svn+svn://[Link]/some_pkg/trunk/#egg=SomeProject # from svn

pip install -e git+[Link] # from a
˓→branch

2.1.10 Installing from other Indexes

Install from an alternate index

pip install --index-url [Link] SomeProject

Search an additional index during install, in addition to PyPI

pip install --extra-index-url [Link] SomeProject

2.1.11 Installing from a local src tree

Installing from local src in Development Mode, i.e. in such a way that the project appears to be installed, but yet is
still editable from the src tree.

pip install -e <path>

You can also install normally from src

14 Chapter 2. Tutorials
 Python Packaging User Guide

pip install <path>

2.1.12 Installing from local archives

Install a particular source archive file.

pip install ./downloads/[Link]

Install from a local directory containing archives (and don’t check PyPI)

pip install --no-index --find-links=[Link] SomeProject

pip install --no-index --find-links=/local/dir/ SomeProject
pip install --no-index --find-links=relative/dir/ SomeProject

2.1.13 Installing from other sources

To install from other data sources (for example Amazon S3 storage) you can create a helper application that presents
the data in a PEP 503 compliant index format, and use the --extra-index-url flag to direct pip to use that index.

./s3helper --port=7777
pip install --extra-index-url [Link] SomeProject

2.1.14 Installing Prereleases

Find pre-release and development versions, in addition to stable versions. By default, pip only finds stable versions.

pip install --pre SomeProject

2.1.15 Installing Setuptools “Extras”

Install setuptools extras.

$ pip install SomePackage[PDF]

$ pip install SomePackage[PDF]==3.0
$ pip install -e .[PDF]==3.0 # editable project in current directory

2.2 Managing Application Dependencies

The package installation tutorial covered the basics of getting set up to install and update Python packages.
However, running these commands interactively can get tedious even for your own personal projects, and things
get even more difficult when trying to set up development environments automatically for projects with multiple
contributors.
This tutorial walks you through the use of Pipenv to manage dependencies for an application. It will show you how to
install and use the necessary tools and make strong recommendations on best practices.

2.2. Managing Application Dependencies 15

Python Packaging User Guide

Keep in mind that Python is used for a great many different purposes, and precisely how you want to manage your
dependencies may change based on how you decide to publish your software. The guidance presented here is most
directly applicable to the development and deployment of network services (including web applications), but is also
very well suited to managing development and testing environments for any kind of project.

Note: This guide is written for Python 3, however, these instructions should also work on Python 2.7.

2.2.1 Installing Pipenv

Pipenv is a dependency manager for Python projects. If you’re familiar with [Link]’ npm or Ruby’s bundler, it
is similar in spirit to those tools. While pip alone is often sufficient for personal use, Pipenv is recommended for
collaborative projects as it’s a higher-level tool that simplifies dependency management for common use cases.
Use pip to install Pipenv:
pip install --user pipenv

Note: This does a user installation to prevent breaking any system-wide packages. If pipenv isn’t available in your
shell after installation, you’ll need to add the user base’s binary directory to your PATH. See Installing to the User Site
for more information.

2.2.2 Installing packages for your project

Pipenv manages dependencies on a per-project basis. To install packages, change into your project’s directory (or just
an empty directory for this tutorial) and run:
cd myproject
pipenv install requests

Pipenv will install the Requests library and create a Pipfile for you in your project’s directory. The Pipfile is used to
track which dependencies your project needs in case you need to re-install them, such as when you share your project
with others. You should get output similar to this (although the exact paths shown will vary):
Creating a Pipfile for this project...
Creating a virtualenv for this project...
Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/[Link]/
˓→Versions/3.6'

New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6

Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd

Installing requests...
Collecting requests
Using cached [Link]
Collecting idna<2.7,>=2.5 (from requests)
Using cached [Link]
Collecting urllib3<1.23,>=1.21.1 (from requests)
Using cached [Link]
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached [Link]
(continues on next page)

16 Chapter 2. Tutorials
 Python Packaging User Guide

(continued from previous page)

Collecting certifi>=2017.4.17 (from requests)
Using cached [Link]
Installing collected packages: idna, urllib3, chardet, certifi, requests
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4
˓→urllib3-1.22

Adding requests to Pipfile's [packages]...

2.2.3 Using installed packages

Now that Requests is installed you can create a simple [Link] file to use it:

import requests

response = [Link]('[Link]

print('Your IP is {0}'.format([Link]()['origin']))

Then you can run this script using pipenv run:

pipenv run python [Link]

You should get output similar to this:

Your IP is [Link]

Using pipenv run ensures that your installed packages are available to your script. It’s also possible to spawn a
new shell that ensures all commands have access to your installed packages with pipenv shell.

2.2.4 Next steps

Congratulations, you now know how to effectively manage dependencies and development environments on a collab-
orative Python project!
If you’re interested in creating and distributing your own Python packages, see the tutorial on packaging and distribut-
ing packages.
Note that when your application includes definitions of Python source packages, they (and
their dependencies) can be added to your pipenv environment with pipenv install -e
<relative-path-to-source-directory> (e.g. pipenv install -e . or pipenv install
-e src).
If you find this particular approach to managing application dependencies isn’t working well for you or your use case,
you may want to explore these other tools and techniques to see if one of them is a better fit:
• pip-tools to build your own custom workflow from lower level pieces like pip-compile and pip-sync
• hatch for opinionated coverage of even more steps in the project management workflow (such as incrementing
versions, tagging releases, and creating new skeleton projects from project templates)
• poetry for a tool comparable in scope to pipenv that focuses more directly on use cases where the repository
being managed is structured as a Python project with a valid [Link] file (by contrast, pipenv
explicitly avoids making the assumption that the application being worked on that’s depending on components
from PyPI will itself support distribution as a pip-installable Python package).

2.2. Managing Application Dependencies 17

Python Packaging User Guide

2.3 Packaging Python Projects

This tutorial walks you through how to package a simple Python project. It will show you how to add the necessary
files and structure to create the package, how to build the package, and how to upload it to the Python Package Index.

2.3.1 A simple project

This tutorial uses a simple project named example_pkg. If you are unfamiliar with Python’s modules and import
packages, take a few minutes to read over the Python documentation for packages and modules. Even if you already
have a project that you want to package up, we recommend following this tutorial as-is using this example package
and then trying with your own package.
To create this project locally, create the following file structure:

/packaging_tutorial
/example_pkg
__init__.py

Once you create this structure, you’ll want to run all of the commands in this tutorial within the top-level folder - so
be sure to cd packaging_tutorial.
You should also edit example_pkg/__init__.py and put the following code in there:

name = "example_pkg"

This is just so that you can verify that it installed correctly later in this tutorial and is not used by PyPI.

2.3.2 Creating the package files

You will now create a handful of files to package up this project and prepare it for distribution. Create the new files
listed below - you will add content to them in the following steps.

/packaging_tutorial
/example_pkg
__init__.py
[Link]
LICENSE
[Link]

2.3.3 Creating [Link]

[Link] is the build script for setuptools. It tells setuptools about your package (such as the name and version) as
well as which code files to include.
Open [Link] and enter the following content. Update the package name to include your username (for example,
example-pkg-theacodes), this ensures that you have a unique package name and that your package doesn’t
conflict with packages uploaded by other people following this tutorial.

import setuptools

with open("[Link]", "r") as fh:

long_description = [Link]()

(continues on next page)

18 Chapter 2. Tutorials
 Python Packaging User Guide

(continued from previous page)

[Link](
name="example-pkg-your-username",
version="0.0.1",
author="Example Author",
author_email="author@[Link]",
description="A small example package",
long_description=long_description,
long_description_content_type="text/markdown",
url="[Link]
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)

setup() takes several arguments. This example package uses a relatively minimal set:
• name is the distribution name of your package. This can be any name as long as only contains letters, numbers,
_ , and -. It also must not already taken on [Link]. Be sure to update this with your username, as this
ensures you won’t try to upload a package with the same name as one which already exists when you upload the
package.
• version is the package version see PEP 440 for more details on versions.
• author and author_email are used to identify the author of the package.
• description is a short, one-sentence summary of the package.
• long_description is a detailed description of the package. This is shown on the package detail package
on the Python Package Index. In this case, the long description is loaded from [Link] which is a common
pattern.
• long_description_content_type tells the index what type of markup is used for the long description.
In this case, it’s Markdown.
• url is the URL for the homepage of the project. For many projects, this will just be a link to GitHub, GitLab,
Bitbucket, or similar code hosting service.
• packages is a list of all Python import packages that should be included in the distribution package. Instead
of listing each package manually, we can use find_packages() to automatically discover all packages and
subpackages. In this case, the list of packages will be example_pkg as that’s the only package present.
• classifiers gives the index and pip some additional metadata about your package. In this case, the package
is only compatible with Python 3, is licensed under the MIT license, and is OS-independent. You should
always include at least which version(s) of Python your package works on, which license your package is
available under, and which operating systems your package will work on. For a complete list of classifiers, see
[Link]
There are many more than the ones mentioned here. See Packaging and distributing projects for more details.

2.3.4 Creating [Link]

Open [Link] and enter the following content. You can customize this if you’d like.

2.3. Packaging Python Projects 19

Python Packaging User Guide

# Example Package

This is a simple example package. You can use

[Github-flavored Markdown]([Link]
to write your content.

2.3.5 Creating a LICENSE

It’s important for every package uploaded to the Python Package Index to include a license. This tells users who install
your package the terms under which they can use your package. For help picking a license, see [Link]
com/. Once you have chosen a license, open LICENSE and enter the license text. For example, if you had chosen the
MIT license:
Copyright (c) 2018 The Python Packaging Authority

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

2.3.6 Generating distribution archives

The next step is to generate distribution packages for the package. These are archives that are uploaded to the Package
Index and can be installed by pip.
Make sure you have the latest versions of setuptools and wheel installed:
python3 -m pip install --user --upgrade setuptools wheel

Tip: IF you have trouble installing these, see the Installing Packages tutorial.

Now run this command from the same directory where [Link] is located:
python3 [Link] sdist bdist_wheel

This command should output a lot of text and once completed should generate two files in the dist directory:
dist/
example_pkg_your_username-[Link]
example_pkg_your_username-[Link]

20 Chapter 2. Tutorials
 Python Packaging User Guide

Note: If you run into trouble here, please copy the output and file an issue over on packaging problems and we’ll do
our best to help you!

The [Link] file is a source archive whereas the .whl file is a built distribution. Newer pip versions preferentially
install built distributions, but will fall back to source archives if needed. You should always upload a source archive
and provide built archives for the platforms your project is compatible with. In this case, our example package is
compatible with Python on any platform so only one built distribution is needed.

2.3.7 Uploading the distribution archives

Finally, it’s time to upload your package to the Python Package Index!
The first thing you’ll need to do is register an account on Test PyPI. Test PyPI is a separate instance of the package
index intended for testing and experimentation. It’s great for things like this tutorial where we don’t necessarily want
to upload to the real index. To register an account, go to [Link] and complete the steps
on that page. You will also need to verify your email address before you’re able to upload any packages. For more
details on Test PyPI, see Using TestPyPI.
Now that you are registered, you can use twine to upload the distribution packages. You’ll need to install Twine:

python3 -m pip install --user --upgrade twine

Once installed, run Twine to upload all of the archives under dist:

python3 -m twine upload --repository-url [Link] dist/*

You will be prompted for the username and password you registered with Test PyPI. After the command completes,
you should see output similar to this:

Uploading distributions to [Link]

Enter your username: [your username]
Enter your password:
Uploading example_pkg_your_username-[Link]
100%|| 4.65k/4.65k [00:01<00:00, 2.88kB/s]
Uploading example_pkg_your_username-[Link]
100%|| 4.25k/4.25k [00:01<00:00, 3.05kB/s]

Once uploaded your package should be viewable on TestPyPI, for example, [Link]
example-pkg-your-username

2.3.8 Installing your newly uploaded package

You can use pip to install your package and verify that it works. Create a new virtualenv (see Installing Packages for
detailed instructions) and install your package from TestPyPI:

python3 -m pip install --index-url [Link] --no-deps example-

˓→pkg-your-username

Make sure to specify your username in the package name!

pip should install the package from Test PyPI and the output should look something like this:

2.3. Packaging Python Projects 21

Python Packaging User Guide

Collecting example-pkg-your-username
Downloading [Link]
˓→[Link]

Installing collected packages: example-pkg-your-username

Successfully installed example-pkg-your-username-0.0.1

Note: This example uses --index-url flag to specify TestPyPI instead of live PyPI. Additionally, it specifies
--no-deps. Since TestPyPI doesn’t have the same packages as the live PyPI, it’s possible that attempting to install
dependencies may fail or install something unexpected. While our example package doesn’t have any dependencies,
it’s a good practice to avoid installing dependencies when using TestPyPI.

You can test that it was installed correctly by importing the module and referencing the name property you put in
__init__.py earlier.
Run the Python interpreter (make sure you’re still in your virtualenv):

python

And then import the module and print out the name property. This should be the same regardless of what you name
you gave your distribution package in [Link] (in this case, example-pkg-your-username) because your
import package is example_pkg.

>>> import example_pkg

>>> example_pkg.name
'example_pkg'

2.3.9 Next steps

Congratulations, you’ve packaged and distributed a Python project!

Keep in mind that this tutorial showed you how to upload your package to Test PyPI, which isn’t a permanent storage.
The Test system occasionally deletes packages and accounts. It is best to use Test PyPI for testing and experiments
like this tutorial.
When you are ready to upload a real package to the Python Package Index you can do much the same as you did in
this tutorial, but with these important differences:
• Choose a memorable and unique name for your package. You don’t have to append your username as you did
in the tutorial.
• Register an account on [Link] - note that these are two separate servers and the login details from the
test server are not shared with the main server.
• Use twine upload dist/* to upload your package and enter your credentials for the account you regis-
tered on the real PyPI.
• Install your package from the real PyPI using pip install [your-package].
At this point if you want to read more on packaging Python libraries here are some things you can do:
• Read more about using setuptools to package libraries in Packaging and distributing projects.
• Read about Packaging binary extensions.
• Consider alternatives to setuptools such as flit, hatch, and poetry.

22 Chapter 2. Tutorials
 Python Packaging User Guide

2.4 Creating Documentation

This section covers the basics of how to create documentation using Sphinx and host the documentation for free in
Read The Docs.

2.4.1 Installing Sphinx

Use pip to install Sphinx:

pip install -U sphinx

For other installation methods, see this installation guide by Sphinx.

2.4.2 Getting Started With Sphinx

Create a doc directory inside your project to hold your documentation:

cd /path/to/project
mkdir docs

Run spinx-quickstart inside the docs directory:

cd docs
sphinx-quickstart

This sets up a source directory, walks you through some basic configurations, and creates an [Link] file as well
as a [Link] file.
You can add some information about your project in [Link], then build them:

make html

For more details on the build process, see this guide by Read The Docs.

2.4.3 Other Sources

For a more detailed guide on how to use Sphinx and reStructuredText, please see this documentation tutorial on
Hitchhiker’s Guide to Python.

2.4. Creating Documentation 23

Python Packaging User Guide

24 Chapter 2. Tutorials
 CHAPTER 3

Guides

Guides are focused on accomplishing a specific task and assume that you are already familiar with the basics of Python
packaging. If you’re looking for an introduction to packaging, see Tutorials.

3.1 Tool recommendations

If you’re familiar with Python packaging and installation, and just want to know what tools are currently recommended,
then here it is.

3.1.1 Application dependency management

Use Pipenv to manage library dependencies when developing Python applications. See Managing Application Depen-
dencies for more details on using pipenv.
Consider other tools such as pip when pipenv does not meet your use case.

3.1.2 Installation tool recommendations

• Use pip to install Python packages from PyPI.12 Depending on how pip is installed, you may need to also install
wheel to get the benefit of wheel caching.3
• Use virtualenv, or venv to isolate application specific dependencies from a shared Python installation.4
• If you’re looking for management of fully integrated cross-platform software stacks, consider:
1 There are some cases where you might choose to use easy_install (from setuptools), e.g. if you need to install from Eggs (which pip

doesn’t support). For a detailed breakdown, see pip vs easy_install.

2 The acceptance of PEP 453 means that pip will be available by default in most installations of Python 3.4 or later. See the rationale section

from PEP 453 as for why pip was chosen.

3 [Link] and virtualenv install wheel, whereas ensurepip and venv do not currently. Also, the common “python-pip” package that’s found in

various linux distros, does not depend on “python-wheel” currently.

4 Beginning with Python 3.4, venv will create virtualenv environments with pip installed, thereby making it an equal alternative to virtualenv.

However, using virtualenv will still be recommended for users that need cross-version consistency.

25
Python Packaging User Guide

– buildout: primarily focused on the web development community

– Spack, Hashdist, or conda: primarily focused on the scientific community.

3.1.3 Packaging tool recommendations

• Use setuptools to define projects and create Source Distributions.56

• Use the bdist_wheel setuptools extension available from the wheel project to create wheels. This is espe-
cially beneficial, if your project contains binary extensions.
• Use twine for uploading distributions to PyPI.

3.1.4 Publishing platform migration

The original Python Package Index implementation (previously hosted at [Link]) has been phased out in
favour of an updated implementation hosted at [Link].
See Migrating to [Link] for more information on the status of the migration, and what settings to change in your
clients.

3.2 Installing packages using pip and virtual environments

This guide discusses how to install packages using pip and a virtual environment manager: either venv for Python 3
or virtualenv for Python 2. These are the lowest-level tools for managing Python packages and are recommended if
higher-level tools do not suit your needs.

Note: This doc uses the term package to refer to a Distribution Package which is different from a Import Package
that which is used to import modules in your Python source code.

3.2.1 Installing pip

pip is the reference Python package manager. It’s used to install and update packages. You’ll need to make sure you
have the latest version of pip installed.

Windows

The Python installers for Windows include pip. You should be able to access pip using:

py -m pip --version
pip 9.0.1 from c:\python36\lib\site-packages (Python 3.6.1)

5 Although you can use pure distutils for many projects, it does not support defining dependencies on other projects and is missing several

convenience utilities for automatically populating distribution metadata correctly that are provided by setuptools. Being outside the standard
library, setuptools also offers a more consistent feature set across different versions of Python, and (unlike distutils), recent versions of
setuptools support all of the modern metadata fields described in Core metadata specifications.
Even for projects that do choose to use distutils, when pip installs such projects directly from source (rather than installing from a prebuilt
wheel file), it will actually build your project using setuptools instead.
6 distribute (a fork of setuptools) was merged back into setuptools in June 2013, thereby making setuptools the default choice for packaging.

26 Chapter 3. Guides
 Python Packaging User Guide

You can make sure that pip is up-to-date by running:

py -m pip install --upgrade pip

Linux and macOS

Debian and most other distributions include a python-pip package, if you want to use the Linux distribution-provided
versions of pip see Installing pip/setuptools/wheel with Linux Package Managers.
You can also install pip yourself to ensure you have the latest version. It’s recommended to use the system pip to
bootstrap a user installation of pip:

python3 -m pip install --user --upgrade pip

Afterwards, you should have the newest pip installed in your user site:

python3 -m pip --version

pip 9.0.1 from $HOME/.local/lib/python3.6/site-packages (python 3.6)

3.2.2 Installing virtualenv

Note: If you are using Python 3.3 or newer, the venv module is the preferred way to create and manage virtual
environments. venv is included in the Python standard library and requires no additional installation. If you are using
venv, you may skip this section.

virtualenv is used to manage Python packages for different projects. Using virtualenv allows you to avoid installing
Python packages globally which could break system tools or other projects. You can install virtualenv using pip.
On macOS and Linux:

python3 -m pip install --user virtualenv

On Windows:

py -m pip install --user virtualenv

3.2.3 Creating a virtual environment

venv (for Python 3) and virtualenv (for Python 2) allow you to manage separate package installations for different
projects. They essentially allow you to create a “virtual” isolated Python installation and install packages into that
virtual installation. When you switch projects, you can simply create a new virtual environment and not have to
worry about breaking the packages installed in the other environments. It is always recommended to use a virtual
environment while developing Python applications.
To create a virtual environment, go to your project’s directory and run venv. If you are using Python 2, replace venv
with virtualenv in the below commands.
On macOS and Linux:

python3 -m venv env

On Windows:

3.2. Installing packages using pip and virtual environments 27

Python Packaging User Guide

py -m venv env

The second argument is the location to create the virtual environment. Generally, you can just create this in your
project and call it env.
venv will create a virtual Python installation in the env folder.

Note: You should exclude your virtual environment directory from your version control system using .gitignore
or similar.

3.2.4 Activating a virtual environment

Before you can start installing or using packages in your virtual environment you’ll need to activate it. Activating a
virtual environment will put the virtual environment-specific python and pip executables into your shell’s PATH.
On macOS and Linux:

source env/bin/activate

On Windows:

.\env\Scripts\activate

You can confirm you’re in the virtual environment by checking the location of your Python interpreter, it should point
to the env directory.
On macOS and Linux:

which python
.../env/bin/python

On Windows:

where python
.../env/bin/[Link]

As long as your virtual environment is activated pip will install packages into that specific environment and you’ll be
able to import and use packages in your Python application.

3.2.5 Leaving the virtual environment

If you want to switch projects or otherwise leave your virtual environment, simply run:

deactivate

If you want to re-enter the virtual environment just follow the same instructions above about activating a virtual
environment. There’s no need to re-create the virtual environment.

3.2.6 Installing packages

Now that you’re in your virtual environment you can install packages. Let’s install the excellent Requests library from
the Python Package Index (PyPI):

28 Chapter 3. Guides
 Python Packaging User Guide

pip install requests

pip should download requests and all of its dependencies and install them:

Collecting requests
Using cached [Link]
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached [Link]
Collecting urllib3<1.23,>=1.21.1 (from requests)
Using cached [Link]
Collecting certifi>=2017.4.17 (from requests)
Using cached [Link]
Collecting idna<2.7,>=2.5 (from requests)
Using cached [Link]
Installing collected packages: chardet, urllib3, certifi, idna, requests
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4
˓→urllib3-1.22

3.2.7 Installing specific versions

pip allows you to specify which version of a package to install using version specifiers. For example, to install a
specific version of requests:

pip install requests==2.18.4

To install the latest 2.x release of requests:

pip install requests>=2.0.0,<3.0.0

To install pre-release versions of packages, use the --pre flag:

pip install --pre requests

3.2.8 Installing extras

Some packages have optional extras. You can tell pip to install these by specifying the extra in brackets:

pip install requests[security]

3.2.9 Installing from source

pip can install a package directly from source, for example:

cd google-auth
pip install .

Additionally, pip can install packages from source in development mode, meaning that changes to the source directory
will immediately affect the installed package without needing to re-install:

pip install --editable .

3.2. Installing packages using pip and virtual environments 29

Python Packaging User Guide

3.2.10 Installing from version control systems

pip can install packages directly from their version control system. For example, you can install directly from a git
repository:

git+[Link]
˓→auth

For more information on supported version control systems and syntax, see pip’s documentation on VCS Support.

3.2.11 Installing from local archives

If you have a local copy of a Distribution Package’s archive (a zip, wheel, or tar file) you can install it directly with
pip:

pip install [Link]

If you have a directory containing archives of multiple packages, you can tell pip to look for packages there and not to
use the Python Package Index (PyPI) at all:

pip install --no-index --find-links=/local/dir/ requests

This is useful if you are installing packages on a system with limited connectivity or if you want to strictly control the
origin of distribution packages.

3.2.12 Using other package indexes

If you want to download packages from a different index than the Python Package Index (PyPI), you can use the
--index-url flag:

pip install --index-url [Link] SomeProject

If you want to allow packages from both the Python Package Index (PyPI) and a separate index, you can use the
--extra-index-url flag instead:

pip install --extra-index-url [Link] SomeProject

3.2.13 Upgrading packages

pip can upgrade packages in-place using the --upgrade flag. For example, to install the latest version of requests
and all of its dependencies:

pip install --upgrade requests

3.2.14 Using requirements files

Instead of installing packages individually, pip allows you to declare all dependencies in a Requirements File. For
example you could create a [Link] file containing:

requests==2.18.4
google-auth==1.1.0

30 Chapter 3. Guides
 Python Packaging User Guide

And tell pip to install all of the packages in this file using the -r flag:

pip install -r [Link]

3.2.15 Freezing dependencies

Pip can export a list of all installed packages and their versions using the freeze command:

pip freeze

Which will output a list of package specifiers such as:

cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
google-auth==1.1.1
idna==2.6
pyasn1==0.3.6
pyasn1-modules==0.1.4
requests==2.18.4
rsa==3.4.2
six==1.11.0
urllib3==1.22

This is useful for creating Requirements Files that can re-create the exact versions of all packages installed in an
environment.

3.3 Installing stand alone command line tools

Many packages have command line entry points. Examples of this type of application are mypy, flake8, pipenv,and
black.
Usually you want to be able to access these from anywhere, but installing packages and their dependencies to the same
global environment can cause version conflicts and break dependencies the operating system has on Python packages.
pipx solves this by creating a virtual environment for each package, while also ensuring that package’s applications are
accessible through a directory that is on your $PATH. This allows each package to be upgraded or uninstalled without
causing conflicts with other packages, and allows you to safely run the program from anywhere.

Note: pipx only works with Python 3.6+.

pipx is installed with pip:

$ pip install --user pipx

$ pipx ensurepath # ensures the path of the CLI application directory is on your
˓→$PATH

Note: You may need to restart your terminal for the path updates to take effect.

Now you can install packages with pipx install and access the package’s entry point(s) from anywhere.

3.3. Installing stand alone command line tools 31

Python Packaging User Guide

$ pipx install PACKAGE

$ ENTRYPOINT_OF_PACKAGE [ARGS]

For example

$ pipx install cowsay

installed package cowsay 2.0, Python 3.6.2+
These binaries are now globally available
- cowsay
done!
$ cowsay moo
___
< moo >
===
\
\
^__^
(oo)\_______
(__)\ )\/ ||----w |
|| ||

To see a list of packages installed with pipx and which CLI applications are available, use pipx list.

$ pipx list
venvs are in /Users/user/.local/pipx/venvs
symlinks to binaries are in /Users/user/.local/bin
package black 18.9b0, Python 3.6.2+
- black
- blackd
package cowsay 2.0, Python 3.6.2+
- cowsay
package mypy 0.660, Python 3.6.2+
- dmypy
- mypy
- stubgen
package nox 2018.10.17, Python 3.6.2+
- nox
- tox-to-nox

To upgrade or uninstall the package

$ pipx upgrade PACKAGE

$ pipx uninstall PACKAGE

pipx can be upgraded or uninstalled with pip

$ pip install -U pipx

$ pip uninstall pipx

pipx also allows you to install and run the latest version of a cli tool in a temporary, ephemeral environment.

$ pipx run PACKAGE [ARGS]

For example

$ pipx run cowsay moooo

To see the full list of commands pipx offers, run

32 Chapter 3. Guides
 Python Packaging User Guide

$ pipx --help

You can learn more about pipx at its homepage, [Link]

3.4 Installing pip/setuptools/wheel with Linux Package Managers

Page Status Incomplete

Last Reviewed 2015-09-17
This section covers how to install pip, setuptools, and wheel using Linux package managers.
If you’re using a Python that was downloaded from [Link], then this section does not apply. See the Requirements
for Installing Packages section instead.
Note that it’s common for the versions of pip, setuptools, and wheel supported by a specific Linux Distribution to be
outdated by the time it’s released to the public, and updates generally only occur for security reasons, not for feature
updates. For certain Distributions, there are additional repositories that can be enabled to provide newer versions. The
repositories we know about are explained below.
Also note that it’s somewhat common for Distributions to apply patches for the sake of security and normalization to
their own standards. In some cases, this can lead to bugs or unexpected behaviors that vary from the original unpatched
versions. When this is known, we will make note of it below.

3.4.1 Fedora

• Fedora 21:
– Python 2:

sudo yum upgrade python-setuptools

sudo yum install python-pip python-wheel

– Python 3: sudo yum install python3 python3-wheel

• Fedora 22:
– Python 2:

sudo dnf upgrade python-setuptools

sudo dnf install python-pip python-wheel

– Python 3: sudo dnf install python3 python3-wheel

To get newer versions of pip, setuptools, and wheel for Python 2, you can enable the PyPA Copr Repo using the Copr
Repo instructions, and then run:

sudo yum|dnf upgrade python-setuptools

sudo yum|dnf install python-pip python-wheel

3.4.2 CentOS/RHEL

CentOS and RHEL don’t offer pip or wheel in their core repositories, although setuptools is installed by default.
To install pip and wheel for the system Python, there are two options:

3.4. Installing pip/setuptools/wheel with Linux Package Managers 33

Python Packaging User Guide

1. Enable the EPEL repository using these instructions. On EPEL 6 and EPEL7, you can install pip like so:

sudo yum install python-pip

On EPEL 7 (but not EPEL 6), you can install wheel like so:

sudo yum install python-wheel

Since EPEL only offers extra, non-conflicting packages, EPEL does not offer setuptools, since it’s in the core
repository.
2. Enable the PyPA Copr Repo using these instructions1 . You can install pip and wheel like so:

sudo yum install python-pip python-wheel

To additionally upgrade setuptools, run:

sudo yum upgrade python-setuptools

To install pip, wheel, and setuptools, in a parallel, non-system environment (using yum) then there are two options:
1. Use the “Sofware Collections” feature to enable a parallel collection that includes pip, setuptools, and wheel.
• For Redhat, see here: [Link]
• For CentOS, see here: [Link]
Be aware that collections may not contain the most recent versions.
2. Enable the IUS repository and install one of the parallel-installable Pythons, along with pip, setuptools, and
wheel, which are kept fairly up to date.
For example, for Python 3.4 on CentOS7/RHEL7:

sudo yum install python34u python34u-wheel

3.4.3 openSUSE

• Python 2:

sudo zypper install python-pip python-setuptools python-wheel

• Python 3:

sudo zypper install python3-pip python3-setuptools python3-wheel

3.4.4 Debian/Ubuntu

• Python 2:

sudo apt install python-pip

• Python 3:

1 Currently, there is no “copr” yum plugin available for CentOS/RHEL, so the only option is to manually place the repo files as described.

34 Chapter 3. Guides
 Python Packaging User Guide

sudo apt install python3-venv python3-pip

Warning: Recent Debian/Ubuntu versions have modified pip to use the “User Scheme” by default, which is a
significant behavior change that can be surprising to some users.

3.4.5 Arch Linux

• Python 2:

sudo pacman -S python2-pip

• Python 3:

sudo pacman -S python-pip

3.5 Installing scientific packages

Contents

• Building from source

• Linux distribution packages
• Windows installers
• macOS installers and package managers
• SciPy distributions
• Spack
• The conda cross-platform package manager

Scientific software tends to have more complex dependencies than most, and it will often have multiple build options
to take advantage of different kinds of hardware, or to interoperate with different pieces of external software.
In particular, NumPy, which provides the basis for most of the software in the scientific Python stack can be configured
to interoperate with different FORTRAN libraries, and can take advantage of different levels of vectorised instructions
available in modern CPUs.
Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the wheel
format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that
on Windows, NumPy binaries are linked against the ATLAS <[Link] BLAS/LAPACK library,
restricted to SSE2 instructions, so they may not provide optimal linear algebra performance.
There are a number of alternative options for obtaining scientific Python libraries (or any other Python libraries that
require a compilation environment to install from source and don’t provide pre-built wheel files on PyPI).

3.5. Installing scientific packages 35

Python Packaging User Guide

3.5.1 Building from source

The same complexity which makes it difficult to distribute NumPy (and many of the projects that depend on it) as
wheel files also make them difficult to build from source yourself. However, for intrepid folks that are willing to spend
the time wrangling compilers and linkers for both C and FORTRAN, building from source is always an option.

3.5.2 Linux distribution packages

For Linux users, the system package manager will often have pre-compiled versions of various pieces of scientific
software, including NumPy and other parts of the scientific Python stack.
If using versions which may be several months old is acceptable, then this is likely to be a good option (just make sure
to allow access to distributions installed into the system Python when using virtual environments).

3.5.3 Windows installers

Many Python projects that don’t (or can’t) currently publish wheel files at least publish Windows installers, either
on PyPI or on their project download page. Using these installers allows users to avoid the need to set up a suitable
environment to build extensions locally.
The extensions provided in these installers are typically compatible with the CPython Windows installers published
on [Link].
For projects which don’t provide their own Windows installers (and even some which do), Christoph Gohlke at the
University of California provides a collection of Windows installers. Many Python users on Windows have reported a
positive experience with these prebuilt versions.
As with Linux system packages, the Windows installers will only install into a system Python installation - they do not
support installation in virtual environments. Allowing access to distributions installed into the system Python when
using virtual environments is a common approach to working around this limitation.
The wheel project also provides a wheel convert subcommand that can convert a Windows bdist_wininst installer to a
wheel.

3.5.4 macOS installers and package managers

Similar to the situation on Windows, many projects (including NumPy) publish macOS installers that are compatible
with the macOS CPython binaries published on [Link].
macOS users also have access to Linux distribution style package managers such as MacPorts. The SciPy site has
more details on using MacPorts to install the scientific Python stack

3.5.5 SciPy distributions

The SciPy site lists several distributions that provide the full SciPy stack to end users in an easy to use and update
format.
Some of these distributions may not be compatible with the standard pip and virtualenv based toolchain.

36 Chapter 3. Guides
 Python Packaging User Guide

3.5.6 Spack

Spack is a flexible package manager designed to support multiple versions, configurations, platforms, and compilers.
It was built to support the needs of large supercomputing centers and scientific application teams, who must often build
software many different ways. Spack is not limited to Python; it can install packages for C, C++, Fortran, R, and
other languages. It is non-destructive; installing a new version of one package does not break existing installations, so
many configurations can coexist on the same system.
Spack offers a simple but powerful syntax that allows users to specify versions and configuration options concisely.
Package files are written in pure Python, and they are templated so that it is easy to swap compilers, dependency
implementations (like MPI), versions, and build options with a single package file. Spack also generates module files
so that packages can be loaded and unloaded from the user’s environment.

3.5.7 The conda cross-platform package manager

Anaconda is a Python distribution published by Anaconda, Inc. It is a stable collection of Open Source packages for
big data and scientific use. As of the 5.0 release of Anaconda, about 200 packages are installed by default, and a total
of 400-500 can be installed and updated from the Anaconda repository.
conda is an open source (BSD licensed) package management system and environment management system included
in Anaconda that allows users to install multiple versions of binary software packages and their dependencies, and
easily switch between them. It is a cross-platform tool working on Windows, macOS, and Linux. Conda can be
used to package up and distribute all kinds of packages, it is not limited to just Python packages. It has full support
for native virtual environments. Conda makes environments first-class citizens, making it easy to create independent
environments even for C libraries. It is written in Python, but is Python-agnostic. Conda manages Python itself as a
package, so that conda update python is possible, in contrast to pip, which only manages Python packages. Conda is
available in Anaconda and Miniconda (an easy-to-install download with just Python and conda).

3.6 Multi-version installs

easy_install allows simultaneous installation of different versions of the same project into a single environment
shared by multiple programs which must require the appropriate version of the project at run time (using
pkg_resources).
For many use cases, virtual environments address this need without the complication of the require directive.
However, the advantage of parallel installations within the same environment is that it works for an environment
shared by multiple applications, such as the system Python in a Linux distribution.
The major limitation of pkg_resources based parallel installation is that as soon as you import pkg_resources
it locks in the default version of everything which is already available on [Link]. This can cause problems, since
setuptools created command line scripts use pkg_resources to find the entry point to execute. This means
that, for example, you can’t use require tests invoked through nose or a WSGI application invoked through
gunicorn if your application needs a non-default version of anything that is available on the standard [Link]
- the script wrapper for the main application will lock in the version that is available by default, so the subsequent
require call in your own code fails with a spurious version conflict.
This can be worked around by setting all dependencies in __main__.__requires__ before importing
pkg_resources for the first time, but that approach does mean that standard command line invocations of the
affected tools can’t be used - it’s necessary to write a custom wrapper script or use python -c '<commmand>'
to invoke the application’s main entry point directly.
Refer to the pkg_resources documentation for more details.

3.6. Multi-version installs 37

Python Packaging User Guide

3.7 Packaging and distributing projects

This section covers the basics of how to configure, package and distribute your own Python projects. It assumes that
you are already familiar with the contents of the Installing Packages page.
The section does not aim to cover best practices for Python project development as a whole. For example, it does not
provide guidance or tool recommendations for version control, documentation, or testing.
For more reference material, see Building and Distributing Packages in the setuptools docs, but note that some advisory
content there may be outdated. In the event of conflicts, prefer the advice in the Python Packaging User Guide.

Contents

• Requirements for packaging and distributing

• Configuring your project
– Initial files

* [Link]
* [Link]
* [Link] / [Link]
* [Link]
* [Link]
* <your package>
– setup() args

* name
* version
* description
* url
* author
* license
* classifiers
* keywords
* project_urls
* packages
* py_modules
* install_requires
* python_requires
* package_data
* data_files
* scripts
* entry_points

38 Chapter 3. Guides
 Python Packaging User Guide

· console_scripts
– Choosing a versioning scheme

* Standards compliance for interoperability

* Scheme choices
· Semantic versioning (preferred)
· Date based versioning
· Serial versioning
· Hybrid schemes

* Pre-release versioning
* Local version identifiers
• Working in “development mode”
• Packaging your project
– Source distributions
– Wheels

* Universal Wheels
* Pure Python Wheels
* Platform Wheels
• Uploading your Project to PyPI
– Create an account
– Upload your distributions

3.7.1 Requirements for packaging and distributing

1. First, make sure you have already fulfilled the requirements for installing packages.
2. Install “twine”1 :

pip install twine

You’ll need this to upload your project distributions to PyPI (see below).

3.7.2 Configuring your project

Initial files

[Link]

The most important file is [Link] which exists at the root of your project directory. For an example, see the
[Link] in the PyPA sample project.
1 Depending on your platform, this may require root or Administrator access. pip is currently considering changing this by making user installs

the default behavior.

3.7. Packaging and distributing projects 39

Python Packaging User Guide

[Link] serves two primary functions:

1. It’s the file where various aspects of your project are configured. The primary feature of [Link] is that it
contains a global setup() function. The keyword arguments to this function are how specific details of your
project are defined. The most relevant arguments are explained in the section below.
2. It’s the command line interface for running various commands that relate to packaging tasks. To get a listing of
available commands, run python [Link] --help-commands.

[Link]

[Link] is an ini file that contains option defaults for [Link] commands. For an example, see the [Link]
in the PyPA sample project.

[Link] / [Link]

All projects should contain a readme file that covers the goal of the project. The most common format is reStructured-
Text with an “rst” extension, although this is not a requirement; multiple variants of Markdown are supported as well
(look at setup()’s long_description_content_type argument).
For an example, see [Link] from the PyPA sample project.

Note: Projects using setuptools 0.6.27+ have standard readme files ([Link], [Link], or README)
included in source distributions by default. The built-in distutils library adopts this behavior beginning in Python 3.7.
Additionally, setuptools 36.4.0+ will include a [Link] if found. If you are using setuptools, you don’t need to
list your readme file in [Link]. Otherwise, include it to be explicit.

[Link]

A [Link] is needed when you need to package additional files that are not automatically included in a source
distribution. To see a list of what’s included by default, see the Specifying the files to distribute section from the
distutils documentation.
For an example, see the [Link] from the PyPA sample project.
For details on writing a [Link] file, see the The [Link] template section from the distutils documen-
tation.

Note: [Link] does not affect binary distributions such as wheels.

[Link]

Every package should include a license file detailing the terms of distribution. In many jurisdictions, packages without
an explicit license can not be legally used or distributed by anyone other than the copyright holder. If you’re unsure
which license to choose, you can use resources such as GitHub’s Choose a License or consult a lawyer.
For an example, see the [Link] from the PyPA sample project.

40 Chapter 3. Guides
 Python Packaging User Guide

<your package>

Although it’s not required, the most common practice is to include your Python modules and packages under a single
top-level package that has the same name as your project, or something very close.
For an example, see the sample package that’s included in the PyPA sample project.

setup() args

As mentioned above, the primary feature of [Link] is that it contains a global setup() function. The keyword
arguments to this function are how specific details of your project are defined.
The most relevant arguments are explained below. Most of the snippets given are taken from the [Link] contained in
the PyPA sample project.

name

name='sample',

This is the name of your project, determining how your project is listed on PyPI. Per PEP 508, valid project names
must:
• Consist only of ASCII letters, digits, underscores (_), hyphens (-), and/or periods (.), and
• Start & end with an ASCII letter or digit.
Comparison of project names is case insensitive and treats arbitrarily-long runs of underscores, hyphens, and/or periods
as equal. For example, if you register a project named cool-stuff, users will be able to download it or declare a
dependency on it using any of the following spellings:

Cool-Stuff
[Link]
COOL_STUFF
CoOl__-.-__sTuFF

version

version='1.2.0',

This is the current version of your project, allowing your users to determine whether or not they have the latest version,
and to indicate which specific versions they’ve tested their own software against.
Versions are displayed on PyPI for each release if you publish your project.
See Choosing a versioning scheme for more information on ways to use versions to convey compatibility information
to your users.
If the project code itself needs run-time access to the version, the simplest way is to keep the version in both setup.
py and your code. If you’d rather not duplicate the value, there are a few ways to manage this. See the “Single-sourcing
the package version” Advanced Topics section.

3.7. Packaging and distributing projects 41

Python Packaging User Guide

description

description='A sample Python project',

long_description=long_description,
long_description_content_type='text/x-rst',

Give a short and long description for your project.

These values will be displayed on PyPI if you publish your project. On [Link], the user interface displays
description in the grey banner and long_description in the section named “Project Description”.
description is also displayed in lists of projects. For example, it’s visible in the search results pages such as
[Link] the front-page lists of trending projects and new releases, and the list of projects
you maintain within your account profile (such as [Link]
A content type can be specified with the long_description_content_type argument, which can be one of
text/plain, text/x-rst, or text/markdown, corresponding to no formatting, reStructuredText (reST), and
the Github-flavored Markdown dialect of Markdown respectively.

url

url='[Link]

Give a homepage URL for your project.

author

author='The Python Packaging Authority',

author_email='pypa-dev@[Link]',

Provide details about the author.

license

license='MIT',

The license argument doesn’t have to indicate the license under which your package is being released, although
you may optionally do so if you want. If you’re using a standard, well-known license, then your main indication can
and should be via the classifiers argument. Classifiers exist for all major open-source licenses.
The “license” argument is more typically used to indicate differences from well-known licenses, or to include your
own, unique license. As a general rule, it’s a good idea to use a standard, well-known license, both to avoid confusion
and because some organizations avoid software whose license is unapproved.
See “Classifier” for some examples of values for license.

classifiers

42 Chapter 3. Guides
 Python Packaging User Guide

classifiers=[
# How mature is this project? Common values are
# 3 - Alpha
# 4 - Beta
# 5 - Production/Stable
'Development Status :: 3 - Alpha',

# Indicate who your project is intended for

'Intended Audience :: Developers',
'Topic :: Software Development :: Build Tools',

# Pick your license as you wish (should match "license" above)

'License :: OSI Approved :: MIT License',

# Specify the Python versions you support here. In particular, ensure

# that you indicate whether you support Python 2, Python 3 or both.
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
],

Provide a list of classifiers that categorize your project. For a full listing, see [Link]
Although the list of classifiers is often used to declare what Python versions a project supports, this information is only
used for searching & browsing projects on PyPI, not for installing projects. To actually restrict what Python versions
a project can be installed on, use the python_requires argument.

keywords

keywords='sample setuptools development',

List keywords that describe your project.

project_urls

project_urls={
'Documentation': '[Link]
'Funding': '[Link]
'Say Thanks!': '[Link]
'Source': '[Link]
'Tracker': '[Link]
},

List additional relevant URLs about your project. This is the place to link to bug trackers, source repositories, or where
to support package development. The string of the key is the exact text that will be displayed on PyPI.

3.7. Packaging and distributing projects 43

Python Packaging User Guide

packages

packages=find_packages(exclude=['contrib', 'docs', 'tests*']),

Set packages to a list of all packages in your project, including their subpackages, sub-subpackages, etc. Al-
though the packages can be listed manually, setuptools.find_packages() finds them automatically. Use the
exclude keyword argument to omit packages that are not intended to be released and installed.

py_modules

py_modules=["six"],

If your project contains any single-file Python modules that aren’t part of a package, set py_modules to a list of the
names of the modules (minus the .py extension) in order to make setuptools aware of them.

install_requires

install_requires=['peppercorn'],

“install_requires” should be used to specify what dependencies a project minimally needs to run. When the project is
installed by pip, this is the specification that is used to install its dependencies.
For more on using “install_requires” see install_requires vs requirements files.

python_requires

If your project only runs on certain Python versions, setting the python_requires argument to the appropriate
PEP 440 version specifier string will prevent pip from installing the project on other Python versions. For example, if
your package is for Python 3+ only, write:

python_requires='>=3',

If your package is for Python 3.3 and up but you’re not willing to commit to Python 4 support yet, write:

python_requires='~=3.3',

If your package is for Python 2.6, 2.7, and all versions of Python 3 starting with 3.3, write:

python_requires='>=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4',

And so on.

Note: Support for this feature is relatively recent. Your project’s source distributions and wheels (see Packaging your
project) must be built using at least version 24.2.0 of setuptools in order for the python_requires argument to be
recognized and the appropriate metadata generated.
In addition, only versions 9.0.0 and higher of pip recognize the python_requires metadata. Users with ear-
lier versions of pip will be able to download & install projects on any Python version regardless of the projects’
python_requires values.

44 Chapter 3. Guides
 Python Packaging User Guide

package_data

package_data={
'sample': ['package_data.dat'],
},

Often, additional files need to be installed into a package. These files are often data that’s closely related to the
package’s implementation, or text files containing documentation that might be of interest to programmers using the
package. These files are called “package data”.
The value must be a mapping from package name to a list of relative path names that should be copied into the package.
The paths are interpreted as relative to the directory containing the package.
For more information, see Including Data Files from the setuptools docs.

data_files

data_files=[('my_data', ['data/data_file'])],

Although configuring package_data is sufficient for most needs, in some cases you may need to place data files outside
of your packages. The data_files directive allows you to do that. It is mostly useful if you need to install files
which are used by other programs, which may be unaware of Python packages.
Each (directory, files) pair in the sequence specifies the installation directory and the files to install there.
The directory must be a relative path (although this may change in the future, see wheel Issue #92). and it is
interpreted relative to the installation prefix (Python’s [Link] for a default installation; site.USER_BASE
for a user installation). Each file name in files is interpreted relative to the [Link] script at the top of the
project source distribution.
For more information see the distutils section on Installing Additional Files.

Note: When installing packages as egg, data_files is not supported. So, if your project uses setuptools,
you must use pip to install it. Alternatively, if you must use python [Link], then you need to pass the
--old-and-unmanageable option.

scripts

Although setup() supports a scripts keyword for pointing to pre-made scripts to install, the recommended approach
to achieve cross-platform compatibility is to use console_scripts entry points (see below).

entry_points

entry_points={
...
},

Use this keyword to specify any plugins that your project provides for any named entry points that may be defined by
your project or others that you depend on.
For more information, see the section on Dynamic Discovery of Services and Plugins from the setuptools docs.
The most commonly used entry point is “console_scripts” (see below).

3.7. Packaging and distributing projects 45

Python Packaging User Guide

console_scripts

entry_points={
'console_scripts': [
'sample=sample:main',
],
},

Use “console_script” entry points to register your script interfaces. You can then let the toolchain handle the work of
turning these interfaces into actual scripts2 . The scripts will be generated during the install of your distribution.
For more information, see Automatic Script Creation from the setuptools docs.

Choosing a versioning scheme

Standards compliance for interoperability

Different Python projects may use different versioning schemes based on the needs of that particular project, but all of
them are required to comply with the flexible public version scheme specified in PEP 440 in order to be supported in
tools and libraries like pip and setuptools.
Here are some examples of compliant version numbers:

1.2.0.dev1 # Development release

1.2.0a1 # Alpha Release
1.2.0b1 # Beta Release
1.2.0rc1 # Release Candidate
1.2.0 # Final Release
1.2.0.post1 # Post Release
15.10 # Date based release
23 # Serial release

To further accommodate historical variations in approaches to version numbering, PEP 440 also defines a comprehen-
sive technique for version normalisation that maps variant spellings of different version numbers to a standardised
canonical form.

Scheme choices

Semantic versioning (preferred)

For new projects, the recommended versioning scheme is based on Semantic Versioning, but adopts a different ap-
proach to handling pre-releases and build metadata.
The essence of semantic versioning is a 3-part [Link] numbering scheme, where the
project author increments:
1. MAJOR version when they make incompatible API changes,
2. MINOR version when they add functionality in a backwards-compatible manner, and
3. MAINTENANCE version when they make backwards-compatible bug fixes.
2 Specifically, the “console_script” approach generates .exe files on Windows, which are necessary because the OS special-cases .exe files.
Script-execution features like PATHEXT and the Python Launcher for Windows allow scripts to be used in many cases, but not all.

46 Chapter 3. Guides
 Python Packaging User Guide

Adopting this approach as a project author allows users to make use of “compatible release” specifiers, where name
~= X.Y requires at least release X.Y, but also allows any later release with a matching MAJOR version.
Python projects adopting semantic versioning should abide by clauses 1-8 of the Semantic Versioning 2.0.0 specifica-
tion.

Date based versioning

Semantic versioning is not a suitable choice for all projects, such as those with a regular time based release cadence
and a deprecation process that provides warnings for a number of releases prior to removal of a feature.
A key advantage of date based versioning is that it is straightforward to tell how old the base feature set of a particular
release is given just the version number.
Version numbers for date based projects typically take the form of [Link] (for example, 12.04, 15.10).

Serial versioning

This is the simplest possible versioning scheme, and consists of a single number which is incremented every release.
While serial versioning is very easy to manage as a developer, it is the hardest to track as an end user, as serial version
numbers convey little or no information regarding API backwards compatibility.

Hybrid schemes

Combinations of the above schemes are possible. For example, a project may combine date based versioning with
serial versioning to create a [Link] numbering scheme that readily conveys the approximate age of a release,
but doesn’t otherwise commit to a particular release cadence within the year.

Pre-release versioning

Regardless of the base versioning scheme, pre-releases for a given final release may be published as:
• zero or more dev releases (denoted with a “.devN” suffix)
• zero or more alpha releases (denoted with a “.aN” suffix)
• zero or more beta releases (denoted with a “.bN” suffix)
• zero or more release candidates (denoted with a “.rcN” suffix)
pip and other modern Python package installers ignore pre-releases by default when deciding which versions of
dependencies to install.

Local version identifiers

Public version identifiers are designed to support distribution via PyPI. Python’s software distribution tools also sup-
port the notion of a local version identifier, which can be used to identify local development builds not intended for
publication, or modified variants of a release maintained by a redistributor.
A local version identifier takes the form <public version identifier>+<local version label>.
For example:

3.7. Packaging and distributing projects 47

Python Packaging User Guide

1.2.0.dev1+hg.5.b11e5e6f0b0b # 5th VCS commmit since 1.2.0.dev1 release

1.2.1+fedora.4 # Package with downstream Fedora patches applied

3.7.3 Working in “development mode”

Although not required, it’s common to locally install your project in “editable” or “develop” mode while you’re
working on it. This allows your project to be both installed and editable in project form.
Assuming you’re in the root of your project directory, then run:

pip install -e .

Although somewhat cryptic, -e is short for --editable, and . refers to the current working directory, so together,
it means to install the current directory (i.e. your project) in editable mode. This will also install any dependencies
declared with “install_requires” and any scripts declared with “console_scripts”. Dependencies will be installed in the
usual, non-editable mode.
It’s fairly common to also want to install some of your dependencies in editable mode as well. For example, supposing
your project requires “foo” and “bar”, but you want “bar” installed from VCS in editable mode, then you could
construct a requirements file like so:

-e .
-e git+[Link]

The first line says to install your project and any dependencies. The second line overrides the “bar” dependency, such
that it’s fulfilled from VCS, not PyPI.
If, however, you want “bar” installed from a local directory in editable mode, the requirements file should look like
this, with the local paths at the top of the file:

-e /path/to/project/bar
-e .

Otherwise, the dependency will be fulfilled from PyPI, due to the installation order of the requirements file. For more
on requirements files, see the Requirements File section in the pip docs. For more on VCS installs, see the VCS
Support section of the pip docs.
Lastly, if you don’t want to install any dependencies at all, you can run:

pip install -e . --no-deps

For more information, see the Development Mode section of the setuptools docs.

3.7.4 Packaging your project

To have your project installable from a Package Index like PyPI, you’ll need to create a Distribution (aka “Package”)
for your project.

Source distributions

Minimally, you should create a Source Distribution:

python [Link] sdist

48 Chapter 3. Guides
 Python Packaging User Guide

A “source distribution” is unbuilt (i.e. it’s not a Built Distribution), and requires a build step when installed by pip.
Even if the distribution is pure Python (i.e. contains no extensions), it still involves a build step to build out the
installation metadata from [Link].

Wheels

You should also create a wheel for your project. A wheel is a built package that can be installed without needing to
go through the “build” process. Installing wheels is substantially faster for the end user than installing from a source
distribution.
If your project is pure Python (i.e. contains no compiled extensions) and natively supports both Python 2 and 3, then
you’ll be creating what’s called a *Universal Wheel* (see section below).
If your project is pure Python but does not natively support both Python 2 and 3, then you’ll be creating a “Pure
Python Wheel” (see section below).
If your project contains compiled extensions, then you’ll be creating what’s called a *Platform Wheel* (see section
below).
Before you can build wheels for your project, you’ll need to install the wheel package:

pip install wheel

Universal Wheels

Universal Wheels are wheels that are pure Python (i.e. contain no compiled extensions) and support Python 2 and 3.
This is a wheel that can be installed anywhere by pip.
To build the wheel:

python [Link] bdist_wheel --universal

You can also permanently set the --universal flag in [Link] (e.g., see sampleproject/[Link]):

[bdist_wheel]
universal=1

Only use the --universal setting, if:

1. Your project runs on Python 2 and 3 with no changes (i.e. it does not require 2to3).
2. Your project does not have any C extensions.
Beware that bdist_wheel does not currently have any checks to warn if you use the setting inappropriately.
If your project has optional C extensions, it is recommended not to publish a universal wheel, because pip will prefer
the wheel over a source installation, and prevent the possibility of building the extension.

Pure Python Wheels

Pure Python Wheels that are not “universal” are wheels that are pure Python (i.e. contain no compiled extensions), but
don’t natively support both Python 2 and 3.
To build the wheel:

python [Link] bdist_wheel

3.7. Packaging and distributing projects 49

Python Packaging User Guide

bdist_wheel will detect that the code is pure Python, and build a wheel that’s named such that it’s usable on any Python
installation with the same major version (Python 2 or Python 3) as the version you used to build the wheel. For details
on the naming of wheel files, see PEP 425.
If your code supports both Python 2 and 3, but with different code (e.g., you use “2to3”) you can run [Link]
bdist_wheel twice, once with Python 2 and once with Python 3. This will produce wheels for each version.

Platform Wheels

Platform Wheels are wheels that are specific to a certain platform like Linux, macOS, or Windows, usually due to
containing compiled extensions.
To build the wheel:

python [Link] bdist_wheel

bdist_wheel will detect that the code is not pure Python, and build a wheel that’s named such that it’s only usable on
the platform that it was built on. For details on the naming of wheel files, see PEP 425.

Note: PyPI currently supports uploads of platform wheels for Windows, macOS, and the multi-distro manylinux1
ABI. Details of the latter are defined in PEP 513.

3.7.5 Uploading your Project to PyPI

When you ran the command to create your distribution, a new directory dist/ was created under your project’s root
directory. That’s where you’ll find your distribution file(s) to upload.

Note: These files are only created when you run the command to create your distribution. This means that any time
you change the source of your project or the configuration in your [Link] file, you will need to rebuild these files
again before you can distribute the changes to PyPI.

Note: Before releasing on main PyPI repo, you might prefer training with the PyPI test site which is cleaned on a
semi regular basis. See Using TestPyPI on how to setup your configuration in order to use it.

Warning: In other resources you may encounter references to using python [Link] register and
python [Link] upload. These methods of registering and uploading a package are strongly discour-
aged as it may use a plaintext HTTP or unverified HTTPS connection on some Python versions, allowing your
username and password to be intercepted during transmission.

Tip: The reStructuredText parser used on PyPI is not Sphinx! Furthermore, to ensure safety of all users, certain kinds
of URLs and directives are forbidden or stripped out (e.g., the .. raw:: directive). Before trying to upload your
distribution, you should check to see if your brief / long descriptions provided in [Link] are valid. You can do
this by following the instructions for the pypa/readme_renderer tool.

50 Chapter 3. Guides
 Python Packaging User Guide

Create an account

First, you need a PyPI user account. You can create an account using the form on the PyPI website.

Note: If you want to avoid entering your username and password when uploading, you can create a $HOME/.
pypirc file with your username and password:

[pypi]
username = <username>
password = <password>

Be aware that this stores your password in plaintext.

Upload your distributions

Once you have an account you can upload your distributions to PyPI using twine.
The process for uploading a release is the same regardless of whether or not the project already exists on PyPI - if it
doesn’t exist yet, it will be automatically created when the first release is uploaded.
For the second and subsequent releases, PyPI only requires that the version number of the new release differ from any
previous releases.

twine upload dist/*

You can see if your package has successfully uploaded by navigating to the URL [Link]
<sampleproject> where sampleproject is the name of your project that you uploaded. It may take a minute
or two for your project to appear on the site.

3.8 Single-sourcing the package version

There are many techniques to maintain a single source of truth for the version number of your project:
1. Read the file in [Link] and parse the version with a regex. Example ( from pip [Link]):

here = [Link]([Link](__file__))

def read(*parts):
with [Link]([Link](here, *parts), 'r') as fp:
return [Link]()

def find_version(*file_paths):
version_file = read(*file_paths)
version_match = [Link](r"^__version__ = ['\"]([^'\"]*)['\"]",
version_file, re.M)
if version_match:
return version_match.group(1)
raise RuntimeError("Unable to find version string.")

setup(
...
(continues on next page)

3.8. Single-sourcing the package version 51

Python Packaging User Guide

(continued from previous page)

version=find_version("package", "__init__.py")
...
)

Note: This technique has the disadvantage of having to deal with complexities of regular expressions.

2. Use an external build tool that either manages updating both locations, or offers an API that both locations can
use.
Few tools you could use, in no particular order, and not necessarily complete: bump2version, changes,
[Link].
3. Set the value to a __version__ global variable in a dedicated module in your project (e.g. [Link]),
then have [Link] read and exec the value into a variable.

version = {}
with open("...sample/[Link]") as fp:
exec([Link](), version)
# later on we use: version['__version__']

Example using this technique: warehouse.

4. Place the value in a simple VERSION text file and have both [Link] and the project code read it.

with open([Link](mypackage_root_dir, 'VERSION')) as version_file:

version = version_file.read().strip()

An advantage with this technique is that it’s not specific to Python. Any tool can read the version.

Warning: With this approach you must make sure that the VERSION file is included in all your source and
binary distributions (e.g. add include VERSION to your [Link]).

5. Set the value in [Link], and have the project code use the pkg_resources API.

import pkg_resources
assert pkg_resources.get_distribution('pip').version == '1.2.0'

Be aware that the pkg_resources API only knows about what’s in the installation metadata, which is not
necessarily the code that’s currently imported.
Example using this technique: setuptools.
6. Set the value to __version__ in sample/__init__.py and import sample in [Link].

import sample
setup(
...
version=sample.__version__
...
)

52 Chapter 3. Guides
 Python Packaging User Guide

Warning: Although this technique is common, beware that it will fail if sample/__init__.py im-
ports packages from install_requires dependencies, which will very likely not be installed yet when
[Link] is run.

7. Keep the version number in the tags of a version control system (Git, Mercurial, etc) instead of in the code, and
automatically extract it from there using setuptools_scm.

3.9 Supporting multiple Python versions

Page Status Incomplete

Last Reviewed 2014-12-24

Contents

• Automated testing and continuous integration

• Tools for single-source Python packages
• What’s in which Python?

FIXME

Useful projects/resources to reference:

- DONE six
- DONE python-future ([Link]
- tox
- DONE Travis and Shining Panda CI (Shining Panda no longer available)
- DONE Appveyor
- DONE Ned Batchelder's "What's in Which Python"
- [Link]
- [Link]
- Lennart Regebro's "Porting to Python 3"
- Greg Hewgill's script to identify the minimum version of Python
required to run a particular script:
[Link]
- the Python 3 porting how to in the main docs
- cross reference to the stable ABI discussion
in the binary extensions topic (once that exists)
- mention version classifiers for distribution metadata

In addition to the work required to create a Python package, it is often necessary that the package must be made
available on different versions of Python. Different Python versions may contain different (or renamed) standard
library packages, and the changes between Python versions 2.x and 3.x include changes in the language syntax.
Performed manually, all the testing required to ensure that the package works correctly on all the target Python versions
(and OSs!) could be very time-consuming. Fortunately, several tools are available for dealing with this, and these will
briefly be discussed here.

3.9. Supporting multiple Python versions 53

Python Packaging User Guide

3.9.1 Automated testing and continuous integration

Several hosted services for automated testing are available. These services will typically monitor your source code
repository (e.g. at Github or Bitbucket) and run your project’s test suite every time a new commit is made.
These services also offer facilities to run your project’s test suite on multiple versions of Python, giving rapid feedback
about whether the code will work, without the developer having to perform such tests themselves.
Wikipedia has an extensive comparison of many continuous-integration systems. There are two hosted services which
when used in conjunction provide automated testing across Linux, Mac and Windows:
• Travis CI provides both a Linux and a macOS environment. The Linux environment is Ubuntu 12.04 LTS Server
Edition 64 bit while the macOS is 10.9.2 at the time of writing.
• Appveyor provides a Windows environment (Windows Server 2012).

TODO Either link to or provide example .yml files for these two
services.

TODO How do we keep the Travis Linux and macOS versions up-to-date in this
document?

Both Travis CI and Appveyor require a YAML-formatted file as specification for the instructions for testing. If any
tests fail, the output log for that specific configuration can be inspected.
For Python projects that are intended to be deployed on both Python 2 and 3 with a single-source strategy, there are a
number of options.

3.9.2 Tools for single-source Python packages

six is a tool developed by Benjamin Peterson for wrapping over the differences between Python 2 and Python 3. The
six package has enjoyed widespread use and may be regarded as a reliable way to write a single-source Python module
that can be use in both Python 2 and 3. The six module can be used from as early as Python 2.5. A tool called
modernize, developed by Armin Ronacher, can be used to automatically apply the code modifications provided by six.
Similar to six, python-future is a package that provides a compatibility layer between Python 2 and Python 3 source
code; however, unlike six, this package aims to provide interoperability between Python 2 and Python 3 with a lan-
guage syntax that matches one of the two Python versions: one may use
• a Python 2 (by syntax) module in a Python 3 project.
• a Python 3 (by syntax) module in a Python 2 project.
Because of the bi-directionality, python-future offers a pathway to converting a Python 2 package to Python 3 syntax
module-by-module. However, in contrast to six, python-future is supported only from Python 2.6. Similar to modern-
ize for six, python-future comes with two scripts called futurize and pasteurize that can be applied to either a
Python 2 module or a Python 3 module respectively.
Use of six or python-future adds an additional runtime dependency to your package: with python-future, the
futurize script can be called with the --stage1 option to apply only the changes that Python 2.6+ already pro-
vides for forward-compatibility to Python 3. Any remaining compatibility problems would require manual changes.

3.9.3 What’s in which Python?

Ned Batchelder provides a list of changes in each Python release for Python 2, Python 3.0-3.3 and Python 3.4-3.6.
These lists may be used to check whether any changes between Python versions may affect your package.

54 Chapter 3. Guides
 Python Packaging User Guide

TODO These lists should be reproduced here (with permission).

TODO The py3 list should be updated to include 3.4

3.10 Dropping support for older Python versions

Dropping support for older Python versions is supported by the standard Core metadata specifications 1.2 specification
via a “Requires-Python” attribute.
Metadata 1.2+ clients, such as Pip 9.0+, will adhere to this specification by matching the current Python runtime and
comparing it with the required version in the package metadata. If they do not match, it will attempt to install the last
package distribution that supported that Python runtime.
This mechanism can be used to drop support for older Python versions, by amending the “Requires-Python” attribute
in the package metadata.
This guide is specifically for users of setuptools, other packaging tools such as flit may offer similar functionality
but users will need to consult relevant documentation.

3.10.1 Requirements

This workflow requires that:

1. The publisher is using the latest version of setuptools,
2. The latest version of twine is used to upload the package,
3. The user installing the package has at least Pip 9.0, or a client that supports the Metadata 1.2 specification.

3.10.2 Defining the Python version required

1. Download the newest version of Setuptools

Ensure that before you generate source distributions or binary distributions, you update Setuptools and install twine.
Steps:

pip install--upgrade setuptools twine

setuptools version should be above 24.0.0.

2. Specify the version ranges for supported Python distributions

You can specify version ranges and exclusion rules, such as at least Python 3. Or, Python 2.7, 3.4 and beyond.
Examples:

Requires-Python: ">=3"
Requires-Python: ">2.7,!=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"

The way to set those values is within the call to setup within your [Link] script. This will insert the Requires-Python
metadata values based on the argument you provide in python_requires.

3.10. Dropping support for older Python versions 55

Python Packaging User Guide

from setuptools import setup

setup(
# Your setup arguments
python_requires='>=2.7', # Your supported Python ranges
)

3. Validating the Metadata before publishing

Within a Python source package (the zip or the tar-gz file you download) is a text file called PKG-INFO.
This file is generated by Distutils or setuptools when it generates the source package. The file contains a set of keys
and values, the list of keys is part of the PyPa standard metadata format.
You can see the contents of the generated file like this:

tar xfO dist/[Link] my-package-1.0.0/PKG-INFO

Validate that the following is in place, before publishing the package:

• If you have upgraded correctly, the Metadata-Version value should be 1.2 or higher.
• The Requires-Python field is set and matches your specification in [Link].

4. Using Twine to publish

Twine has a number of advantages, apart from being faster it is now the supported method for publishing packages.
Make sure you are using the newest version of Twine, at least 1.9.

3.10.3 Dropping a Python release

Once you have published a package with the Requires-Python metadata, you can then make a further update removing
that Python runtime from support.
It must be done in this order for the automated fail-back to work.
For example, you published the Requires-Python: “>=2.7” as version 1.0.0 of your package.
If you were then to update the version string to “>=3.5”, and publish a new version 2.0.0 of your package, any users
running Pip 9.0+ from version 2.7 will have version 1.0.0 of the package installed, and any >=3.5 users will receive
version 2.0.0.

3.11 Packaging binary extensions

Page Status Incomplete

Last Reviewed 2013-12-08
One of the features of the CPython reference interpreter is that, in addition to allowing the execution of Python code, it
also exposes a rich C API for use by other software. One of the most common uses of this C API is to create importable
C extensions that allow things which aren’t always easy to achieve in pure Python code.

56 Chapter 3. Guides
 Python Packaging User Guide

Contents

• An overview of binary extensions

– Use cases
– Disadvantages
– Alternatives to handcoded accelerator modules
– Alternatives to handcoded wrapper modules
– Alternatives for low level system access
• Implementing binary extensions
• Building binary extensions
– Binary extensions for Windows
– Binary extensions for Linux
– Binary extensions for macOS
• Publishing binary extensions
• Additional resources
– Cross-platform wheel generation with scikit-build
– Introduction to C/C++ extension modules

3.11.1 An overview of binary extensions

Use cases

The typical use cases for binary extensions break down into just three conventional categories:
• accelerator modules: these modules are completely self-contained, and are created solely to run faster than the
equivalent pure Python code runs in CPython. Ideally, accelerator modules will always have a pure Python
equivalent to use as a fallback if the accelerated version isn’t available on a given system. The CPython standard
library makes extensive use of accelerator modules.
• wrapper modules: these modules are created to expose existing C interfaces to Python code. They may either
expose the underlying C interface directly, or else expose a more “Pythonic” API that makes use of Python
language features to make the API easier to use. The CPython standard library makes extensive use of wrapper
modules.
• low level system access: these modules are created to access lower level features of the CPython runtime, the
operating system, or the underlying hardware. Through platform specific code, extension modules may achieve
things that aren’t possible in pure Python code. A number of CPython standard library modules are written in C
in order to access interpreter internals that aren’t exposed at the language level.
One particularly notable feature of C extensions is that, when they don’t need to call back into the interpreter run-
time, they can release CPython’s global interpreter lock around long-running operations (regardless of whether
those operations are CPU or IO bound).
Not all extension modules will fit neatly into the above categories. The extension modules included with NumPy, for
example, span all three use cases - they move inner loops to C for speed reasons, wrap external libraries written in C,
FORTRAN and other languages, and use low level system interfaces for both CPython and the underlying operation

3.11. Packaging binary extensions 57

Python Packaging User Guide

system to support concurrent execution of vectorised operations and to tightly control the exact memory layout of
created objects.

Disadvantages

The main disadvantage of using binary extensions is the fact that it makes subsequent distribution of the software more
difficult. One of the advantages of using Python is that it is largely cross platform, and the languages used to write
extension modules (typically C or C++, but really any language that can bind to the CPython C API) typically require
that custom binaries be created for different platforms.
This means that binary extensions:
• require that end users be able to either build them from source, or else that someone publish pre-built binaries
for common platforms
• may not be compatible with different builds of the CPython reference interpreter
• often will not work correctly with alternative interpreters such as PyPy, IronPython or Jython
• if handcoded, make maintenance more difficult by requiring that maintainers be familiar not only with Python,
but also with the language used to create the binary extension, as well as with the details of the CPython C API.
• if a pure Python fallback implementation is provided, make maintenance more difficult by requiring that changes
be implemented in two places, and introducing additional complexity in the test suite to ensure both versions
are always executed.
Another disadvantage of relying on binary extensions is that alternative import mechanisms (such as the ability to
import modules directly from zipfiles) often won’t work for extension modules (as the dynamic loading mechanisms
on most platforms can only load libraries from disk).

Alternatives to handcoded accelerator modules

When extension modules are just being used to make code run faster (after profiling has identified the code where the
speed increase is worth additional maintenance effort), a number of other alternatives should also be considered:
• look for existing optimised alternatives. The CPython standard libary includes a number of optimised data
structures and algorithms (especially in the builtins and the collections and itertools modules). The
Python Package Index also offers additional alternatives. Sometimes, the appropriate choice of standard library
or third party module can avoid the need to create your own accelerator module.
• for long running applications, the JIT compiled PyPy interpreter may offer a suitable alternative to the standard
CPython runtime. The main barrier to adopting PyPy is typically reliance on other binary extension modules
- while PyPy does emulate the CPython C API, modules that rely on that cause problems for the PyPy JIT,
and the emulation layer can often expose latent defects in extension modules that CPython currently tolerates
(frequently around reference counting errors - an object having one live reference instead of two often won’t
break anything, but no references instead of one is a major problem).
• Cython is a mature static compiler that can compile most Python code to C extension modules. The initial
compilation provides some speed increases (by bypassing the CPython interpreter layer), and Cython’s optional
static typing features can offer additional opportunities for speed increases. Using Cython still has the disad-
vantage of increasing the complexity of distributing the resulting application, but has the benefit of having a
reduced barrier to entry for Python programmers (relative to other languages like C or C++).
• Numba is a newer tool, created by members of the scientific Python community, that aims to leverage LLVM
to allow selective compilation of pieces of a Python application to native machine code at runtime. It requires
that LLVM be available on the system where the code is running, but can provide significant speed increases,
especially for operations that are amenable to vectorisation.

58 Chapter 3. Guides
 Python Packaging User Guide

Alternatives to handcoded wrapper modules

The C ABI (Application Binary Interface) is a common standard for sharing functionality between multiple applica-
tions. One of the strengths of the CPython C API (Application Programming Interface) is allowing Python users to
tap into that functionality. However, wrapping modules by hand is quite tedious, so a number of other alternative
approaches should be considered.
The approaches described below don’t simplify the distribution case at all, but they can significantly reduce the main-
tenance burden of keeping wrapper modules up to date.
• In addition to being useful for the creation of accelerator modules, Cython is also useful for creating wrapper
modules. It still involves wrapping the interfaces by hand, however, so may not be a good choice for wrapping
large APIs.
• cffi is a project created by some of the PyPy developers to make it straightforward for developers that already
know both Python and C to expose their C modules to Python applications. It also makes it relatively straight-
forward to wrap a C module based on its header files, even if you don’t know C yourself.
One of the key advantages of cffi is that it is compatible with the PyPy JIT, allowing CFFI wrapper modules
to participate fully in PyPy’s tracing JIT optimisations.
• SWIG is a wrapper interface generator that allows a variety of programming languages, including Python, to
interface with C and C++ code.
• The standard library’s ctypes module, while useful for getting access to C level interfaces when header infor-
mation isn’t available, suffers from the fact that it operates solely at the C ABI level, and thus has no automatic
consistency checking between the interface actually being exported by the library and the one declared in the
Python code. By contrast, the above alternatives are all able to operate at the C API level, using C header files
to ensure consistency between the interface exported by the library being wrapped and the one expected by the
Python wrapper module. While cffi can operate directly at the C ABI level, it suffers from the same interface
inconsistency problems as ctypes when it is used that way.

Alternatives for low level system access

For applications that need low level system access (regardless of the reason), a binary extension module often is
the best way to go about it. This is particularly true for low level access to the CPython runtime itself, since some
operations (like releasing the Global Interpreter Lock) are simply invalid when the interpreter is running code, even if
a module like ctypes or cffi is used to obtain access to the relevant C API interfaces.
For cases where the extension module is manipulating the underlying operating system or hardware (rather than the
CPython runtime), it may sometimes be better to just write an ordinary C library (or a library in another systems
programming language like C++ or Rust that can export a C compatible ABI), and then use one of the wrapping
techniques described above to make the interface available as an importable Python module.

3.11.2 Implementing binary extensions

The CPython Extending and Embedding guide includes an introduction to writing a custom extension module in C.

mention the stable ABI (3.2+, link to the CPython C API docs)
mention the module lifecycle
mention the challenges of shared static state and subinterpreters
mention the implications of the GIL for extension modules
mention the memory allocation APIs in 3.4+

mention again that all this is one of the reasons why you probably
*don't* want to handcode your extension modules :)

3.11. Packaging binary extensions 59

Python Packaging User Guide

3.11.3 Building binary extensions

Binary extensions for Windows

Before it is possible to build a binary extension, it is necessary to ensure that you have a suitable compiler available.
On Windows, Visual C is used to build the official CPython interpreter, and should be used to build compatible binary
extensions.
Python 2.7 used Visual Studio 2008, Python 3.3 and 3.4 used Visual Studio 2010, and Python 3.5+ uses Visual Studio
2015 or later. Unfortunately, older versions of Visual Studio are no longer easily available from Microsoft, so for
versions of Python prior to 3.5, the compilers must be obtained differently if you do not already have a copy of the
relevant version of Visual Studio.
To set up a build environment for binary extensions, the steps are as follows:
For Python 2.7
1. Install “Visual C++ Compiler Package for Python 2.7”, which is available from Microsoft’s website.
2. Use (a recent version of) setuptools in your [Link] (pip will do this for you, in any case).
3. Done.
For Python 3.4
1. Install “Windows SDK for Windows 7 and .NET Framework 4” (v7.1), which is available from
Microsoft’s website.
2. Work from an SDK command prompt (with the environment variables set, and the SDK on PATH).
3. Set DISTUTILS_USE_SDK=1
4. Done.
For Python 3.5
1. Install Visual Studio 2015 Community Edition (or any later version, when these are released).
2. Done.
Note that from Python 3.5 onwards, Visual Studio works in a backward compatible way, which means that any future
version of Visual Studio will be able to build Python extensions for all Python versions from 3.5 onwards.
Building with the recommended compiler on Windows ensures that a compatible C library is used throughout the
Python process.

Binary extensions for Linux

Linux binaries must use a sufficiently old glibc to be compatible with older distributions. The manylinux Docker
images provide a build environment with a glibc old enough to support most current Linux distributions on common
architectures.

Binary extensions for macOS

Binary compatibility on macOS is determined by the target minimum deployment system, e.g. 10.9, which is of-
ten specified with the MACOSX_DEPLOYMENT_TARGET environmental variable when building binaries on macOS.
When building with setuptools / distutils, the deployment target is specified with the flag --plat-name, e.g.
macosx-10.9-x86_64. For common deployment targets for macOS Python distributions, see the MacPython
Spinning Wheels wiki.

60 Chapter 3. Guides
 Python Packaging User Guide

3.11.4 Publishing binary extensions

For interim guidance on this topic, see the discussion in this issue.

FIXME

cover publishing as wheel files on PyPI or a custom index server

cover creation of Windows and macOS installers
cover weak linking
mention the fact that Linux distros have a requirement to build from
source in their own build systems, so binary-only releases are strongly
discouraged

3.11.5 Additional resources

Cross-platform development and distribution of extension modules is a complex topic, so this guide focuses primarily
on providing pointers to various tools that automate dealing with the underlying technical challenges. The additional
resources in this section are instead intended for developers looking to understand more about the underlying binary
interfaces that those systems rely on at runtime.

Cross-platform wheel generation with scikit-build

The scikit-build package helps abstract cross-platform build operations and provides additional capabilities when
creating binary extension packages. Additional documentation is also available on the C runtime, compiler, and build
system generator for Python binary extension modules.

Introduction to C/C++ extension modules

For a more in depth explanation of how extension modules are used by CPython on a Debian system, see the following
articles:
• What are (c)python extension modules?
• Releasing the gil
• Writing cpython extension modules using C++

3.12 Supporting Windows using Appveyor

Page Status Incomplete

Last Reviewed 2015-12-03
This section covers how to use the free Appveyor continuous integration service to provide Windows support for your
project. This includes testing the code on Windows, and building Windows-targeted binaries for projects that use C
extensions.

Contents

• Background
• Setting up

3.12. Supporting Windows using Appveyor 61

Python Packaging User Guide

• Adding Appveyor support to your project

– [Link]
– Support script
– Access to the built wheels
• Additional notes
– Testing with tox
– Automatically uploading wheels
– External dependencies
– Support scripts

3.12.1 Background

Many projects are developed on Unix by default, and providing Windows support can be a challenge, because setting
up a suitable Windows test environment is non-trivial, and may require buying software licenses.
The Appveyor service is a continuous integration service, much like the better-known Travis service that is commonly
used for testing by projects hosted on Github. However, unlike Travis, the build workers on Appveyor are Windows
hosts and have the necessary compilers installed to build Python extensions.
Windows users typically do not have access to a C compiler, and therefore are reliant on projects that use C extensions
distributing binary wheels on PyPI in order for the distribution to be installable via pip install <dist>. By
using Appveyor as a build service (even if not using it for testing) it is possible for projects without a dedicated
Windows environment to provide Windows-targeted binaries.

3.12.2 Setting up

In order to use Appveyor to build Windows wheels for your project, you must have an account on the service. Instruc-
tions on setting up an account are given in the Appveyor documentation. The free tier of account is perfectly adequate
for open source projects.
Appveyor provides integration with Github and Bitbucket, so as long as your project is hosted on one of those two
services, setting up Appveyor integration is straightforward.
Once you have set up your Appveyor account and added your project, Appveyor will automatically build your project
each time a commit occurs. This behaviour will be familiar to users of Travis.

3.12.3 Adding Appveyor support to your project

In order to define how Appveyor should build your project, you need to add an [Link] file to your project.
The full details of what can be included in the file are covered in the Appveyor documentation. This guide will provide
the details necessary to set up wheel builds.
Appveyor includes by default all of the compiler toolchains needed to build extensions for Python. For Python 2.7,
3.5+ and 32-bit versions of 3.3 and 3.4, the tools work out of the box. But for 64-bit versions of Python 3.3 and 3.4,
there is a small amount of additional configuration needed to let distutils know where to find the 64-bit compilers.
(From 3.5 onwards, the version of Visual Studio used includes 64-bit compilers with no additional setup).

62 Chapter 3. Guides
 Python Packaging User Guide

[Link]

1 environment:
2

3 matrix:
4

5 # For Python versions available on Appveyor, see

6 # [Link]
7 # The list here is complete (excluding Python 2.6, which
8 # isn't covered by this document) at the time of writing.
9

10 - PYTHON: "C:\\Python27"
11 - PYTHON: "C:\\Python33"
12 - PYTHON: "C:\\Python34"
13 - PYTHON: "C:\\Python35"
14 - PYTHON: "C:\\Python27-x64"
15 - PYTHON: "C:\\Python33-x64"
16 DISTUTILS_USE_SDK: "1"
17 - PYTHON: "C:\\Python34-x64"
18 DISTUTILS_USE_SDK: "1"
19 - PYTHON: "C:\\Python35-x64"
20 - PYTHON: "C:\\Python36-x64"
21

22 install:
23 # We need wheel installed to build wheels
24 - "%PYTHON%\\[Link] -m pip install wheel"
25

26 build: off
27

28 test_script:
29 # Put your test command here.
30 # If you don't need to build C extensions on 64-bit Python 3.3 or 3.4,
31 # you can remove "[Link]" from the front of the command, as it's
32 # only needed to support those cases.
33 # Note that you must use the environment variable %PYTHON% to refer to
34 # the interpreter you're using - Appveyor does not do anything special
35 # to put the Python version you want to use on PATH.
36 - "[Link] %PYTHON%\\[Link] [Link] test"
37

38 after_test:
39 # This step builds your wheels.
40 # Again, you only need [Link] if you're building C extensions for
41 # 64-bit Python 3.3/3.4. And you need to use %PYTHON% to get the correct
42 # interpreter
43 - "[Link] %PYTHON%\\[Link] [Link] bdist_wheel"
44

45 artifacts:
46 # bdist_wheel puts your built wheel in the dist directory
47 - path: dist\*
48

49 #on_success:
50 # You can use this step to upload your artifacts to a public website.
51 # See Appveyor's documentation for more details. Or you can simply
52 # access your wheels from the Appveyor "artifacts" tab for your build.

This file can be downloaded from here.

The [Link] file must be located in the root directory of your project. It is in YAML format, and consists of

3.12. Supporting Windows using Appveyor 63

Python Packaging User Guide

a number of sections.
The environment section is the key to defining the Python versions for which your wheels will be created. Ap-
pveyor comes with Python 2.6, 2.7, 3.3, 3.4 and 3.5 installed, in both 32-bit and 64-bit builds. The example file builds
for all of these environments except Python 2.6. Installing for Python 2.6 is more complex, as it does not come with
pip included. We don’t support 2.6 in this document (as Windows users still using Python 2 are generally able to move
to Python 2.7 without too much difficulty).
The install section uses pip to install any additional software that the project may require. The only requirement
for building wheels is the wheel project, but projects may wish to customise this code in certain circumstances (for
example, to install additional build packages such as Cython, or test tools such as tox).
The build section simply switches off builds - there is no build step needed for Python, unlike languages like C#.
The main sections that will need to be tailored to your project are test_script and after_test.
The test_script section is where you will run your project’s tests. The supplied file runs your test suite using
[Link] test. If you are only interested in building wheels, and not in running your tests on Windows, you can
replace this section with a dummy command such as echo Skipped Tests. You may wish to use another test
tool, such as nose or [Link]. Or you may wish to use a test driver like tox - however if you are using tox there
are some additional configuration changes you will need to consider, which are described below.
The after_test runs once your tests have completed, and so is where the wheels should be built. Assuming your
project uses the recommended tools (specifically, setuptools) then the [Link] bdist_wheel command
will build your wheels.
Note that wheels will only be built if your tests succeed. If you expect your tests to fail on Windows, you can skip
them as described above.

Support script

The [Link] file relies on a single support script, which sets up the environment to use the SDK compiler
for 64-bit builds on Python 3.3 and 3.4. For projects which do not need a compiler, or which don’t support 3.3 or 3.4
on 64-bit Windows, only the [Link] file is needed.
[Link] is a Windows batch script that runs a single command in an environment with the appropriate compiler for
the selected Python version. All you need to do is to set the single environment variable DISTUTILS_USE_SDK to
a value of 1 and the script does the rest. It sets up the SDK needed for 64-bit builds of Python 3.3 or 3.4, so don’t set
the environment variable for any other builds.
You can simply download the batch file and include it in your project unchanged.

Access to the built wheels

When your build completes, the built wheels will be available from the Appveyor control panel for your project. They
can be found by going to the build status page for each build in turn. At the top of the build output there is a series
of links, one of which is “Artifacts”. That page will include a list of links to the wheels for that Python version /
architecture. You can download those wheels and upload them to PyPI as part of your release process.

3.12.4 Additional notes

Testing with tox

Many projects use the Tox tool to run their tests. It ensures that tests are run in an isolated environment using the exact
files that will be distributed by the project.

64 Chapter 3. Guides
 Python Packaging User Guide

In order to use tox on Appveyor there are a couple of additional considerations (in actual fact, these issues are not
specific to Appveyor, and may well affect other CI systems).
1. By default, tox only passes a chosen subset of environment variables to the test processes. Because
distutils uses environment variables to control the compiler, this “test isolation” feature will cause the
tests to use the wrong compiler by default.
To force tox to pass the necessary environment variables to the subprocess, you need to set the tox configura-
tion option passenv to list the additional environment variables to be passed to the subprocess. For the SDK
compilers, you need
• DISTUTILS_USE_SDK
• MSSdk
• INCLUDE
• LIB
The passenv option can be set in your [Link], or if you prefer to avoid adding Windows-
specific settings to your general project files, it can be set by setting the TOX_TESTENV_PASSENV
environment variable. The supplied [Link] script does this by default whenever
DISTUTILS_USE_SDK is set.
2. When used interactively, tox allows you to run your tests against multiple environments (often, this means
multiple Python versions). This feature is not as useful in a CI environment like Travis or Appveyor, where
all tests are run in isolated environments for each configuration. As a result, projects often supply an argument
-e ENVNAME to tox to specify which environment to use (there are default environments for most versions of
Python).
However, this does not work well with a Windows CI system like Appveyor, where there are (for ex-
ample) two installations of Python 3.4 (32-bit and 64-bit) available, but only one py34 environment
in tox.
In order to run tests using tox, therefore, projects should probably use the default py environment
in tox, which uses the Python interpreter that was used to run tox. This will ensure that when
Appveyor runs the tests, they will be run with the configured interpreter.
In order to support running under the py environment, it is possible that projects with complex tox
configurations might need to modify their [Link] file. Doing so is, however, outside the scope of
this document.

Automatically uploading wheels

It is possible to request Appveyor to automatically upload wheels. There is a deployment step available in
[Link] that can be used to (for example) copy the built artifacts to a FTP site, or an Amazon S3 instance.
Documentation on how to do this is included in the Appveyor guides.
Alternatively, it would be possible to add a twine upload step to the build. The supplied [Link] does
not do this, as it is not clear that uploading new wheels after every commit is desirable (although some projects may
wish to do this).

External dependencies

The supplied scripts will successfully build any distribution that does not rely on 3rd party external libraries for the
build.
It is possible to add steps to the [Link] configuration (typically in the “install” section) to download and/or
build external libraries needed by the distribution. And if needed, it is possible to add extra configuration for the build

3.12. Supporting Windows using Appveyor 65

 Python Packaging User Guide

to supply the location of these libraries to the compiler. However, this level of configuration is beyond the scope of
this document.

Support scripts

For reference, the SDK setup support script is listed here:

appveyor-sample/[Link]

1 @echo off
2 :: To build extensions for 64 bit Python 3, we need to configure environment
3 :: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
4 :: MS Windows SDK for Windows 7 and .NET Framework 4
5 ::
6 :: More details at:
7 :: [Link]
8

9 IF "%DISTUTILS_USE_SDK%"=="1" (
10 ECHO Configuring environment to build with MSVC on a 64bit architecture
11 ECHO Using Windows SDK 7.1
12 "C:\Program Files\Microsoft SDKs\Windows\v7.1\Setup\[Link]" -q -
˓→version:v7.1

13 CALL "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\[Link]" /x64 /release

14 SET MSSdk=1
15 REM Need the following to allow tox to see the SDK compiler
16 SET TOX_TESTENV_PASSENV=DISTUTILS_USE_SDK MSSdk INCLUDE LIB
17 ) ELSE (
18 ECHO Using default MSVC build environment
19 )
20

21 CALL %*

3.13 Packaging namespace packages

Namespace packages allow you to split the sub-packages and modules within a single package across multiple, sepa-
rate distribution packages (referred to as distributions in this document to avoid ambiguity). For example, if you have
the following package structure:

mynamespace/
__init__.py
subpackage_a/
__init__.py
...
subpackage_b/
__init__.py
...
module_b.py
[Link]

And you use this package in your code like so:

from mynamespace import subpackage_a

from mynamespace import subpackage_b

Then you can break these sub-packages into two separate distributions:

66 Chapter 3. Guides
 Python Packaging User Guide

mynamespace-subpackage-a/
[Link]
mynamespace/
subpackage_a/
__init__.py

mynamespace-subpackage-b/
[Link]
mynamespace/
subpackage_b/
__init__.py
module_b.py

Each sub-package can now be separately installed, used, and versioned.

Namespace packages can be useful for a large collection of loosely-related packages (such as a large corpus of
client libraries for multiple products from a single company). However, namespace packages come with several
caveats and are not appropriate in all cases. A simple alternative is to use a prefix on all of your distributions such
as import mynamespace_subpackage_a (you could even use import mynamespace_subpackage_a
as subpackage_a to keep the import object short).

3.13.1 Creating a namespace package

There are currently three different approaches to creating namespace packages:

1. Use native namespace packages. This type of namespace package is defined in PEP 420 and is available in
Python 3.3 and later. This is recommended if packages in your namespace only ever need to support Python 3
and installation via pip.
2. Use pkgutil-style namespace packages. This is recommended for new packages that need to support Python 2
and 3 and installation via both pip and python [Link] install.
3. Use pkg_resources-style namespace packages. This method is recommended if you need compatibility with
packages already using this method or if your package needs to be zip-safe.

Warning: While native namespace packages and pkgutil-style namespace packages are largely compatible,
pkg_resources-style namespace packages are not compatible with the other methods. It’s inadvisable to use differ-
ent methods in different distributions that provide packages to the same namespace.

Native namespace packages

Python 3.3 added implicit namespace packages from PEP 420. All that is required to create a native namespace
package is that you just omit __init__.py from the namespace package directory. An example file structure:

[Link]
mynamespace/
# No __init__.py here.
subpackage_a/
# Sub-packages have __init__.py.
__init__.py
[Link]

3.13. Packaging namespace packages 67

Python Packaging User Guide

It is extremely important that every distribution that uses the namespace package omits the __init__.py or uses
a pkgutil-style __init__.py. If any distribution does not, it will cause the namespace logic to fail and the other
sub-packages will not be importable.
Because mynamespace doesn’t contain an __init__.py, setuptools.find_packages() won’t find the
sub-package. You must explicitly list all packages in your [Link]. For example:

from setuptools import setup

setup(
name='mynamespace-subpackage-a',
...
packages=['mynamespace.subpackage_a']
)

A complete working example of two native namespace packages can be found in the native namespace package
example project.

Note: Because native and pkgutil-style namespace packages are largely compatible, you can use native namespace
packages in the distributions that only support Python 3 and pkgutil-style namespace packages in the distributions that
need to support Python 2 and 3.

pkgutil-style namespace packages

Python 2.3 introduced the pkgutil module and the extend_path function. This can be used to declare namespace
packages that need to be compatible with both Python 2.3+ and Python 3. This is the recommended approach for the
highest level of compatibility.
To create a pkgutil-style namespace package, you need to provide an __init__.py file for the namespace package:

[Link]
mynamespace/
__init__.py # Namespace package __init__.py
subpackage_a/
__init__.py # Sub-package __init__.py
[Link]

The __init__.py file for the namespace package needs to contain only the following:

__path__ = __import__('pkgutil').extend_path(__path__, __name__)

Every distribution that uses the namespace package must include an identical __init__.py. If any distribution
does not, it will cause the namespace logic to fail and the other sub-packages will not be importable. Any additional
code in __init__.py will be inaccessible.
A complete working example of two pkgutil-style namespace packages can be found in the pkgutil namespace example
project.

pkg_resources-style namespace packages

Setuptools provides the pkg_resources.declare_namespace function and the namespace_packages argument to

setup(). Together these can be used to declare namespace packages. While this approach is no longer recom-
mended, it is widely present in most existing namespace packages. If you are creating a new distribution within

68 Chapter 3. Guides
 Python Packaging User Guide

an existing namespace package that uses this method then it’s recommended to continue using this as the different
methods are not cross-compatible and it’s not advisable to try to migrate an existing package.
To create a pkg_resources-style namespace package, you need to provide an __init__.py file for the namespace
package:

[Link]
mynamespace/
__init__.py # Namespace package __init__.py
subpackage_a/
__init__.py # Sub-package __init__.py
[Link]

The __init__.py file for the namespace package needs to contain only the following:

__import__('pkg_resources').declare_namespace(__name__)

Every distribution that uses the namespace package must include an identical __init__.py. If any distribution
does not, it will cause the namespace logic to fail and the other sub-packages will not be importable. Any additional
code in __init__.py will be inaccessible.

Note: Some older recommendations advise the following in the namespace package __init__.py:

try:
__import__('pkg_resources').declare_namespace(__name__)
except ImportError:
__path__ = __import__('pkgutil').extend_path(__path__, __name__)

The idea behind this was that in the rare case that setuptools isn’t available packages would fall-back to the pkgutil-
style packages. This isn’t advisable because pkgutil and pkg_resources-style namespace packages are not cross-
compatible. If the presence of setuptools is a concern then the package should just explicitly depend on setuptools via
install_requires.

Finally, every distribution must provide the namespace_packages argument to setup() in [Link]. For
example:

from setuptools import find_packages, setup

setup(
name='mynamespace-subpackage-a',
...
packages=find_packages()
namespace_packages=['mynamespace']
)

A complete working example of two pkg_resources-style namespace packages can be found in the pkg_resources
namespace example project.

3.14 Creating and discovering plugins

Often when creating a Python application or library you’ll want the ability to provide customizations or extra fea-
tures via plugins. Because Python packages can be separately distributed, your application or library may want to
automatically discover all of the plugins available.
There are three major approaches to doing automatic plugin discovery:

3.14. Creating and discovering plugins 69

Python Packaging User Guide

1. Using naming convention.

2. Using namespace packages.
3. Using package metadata.

3.14.1 Using naming convention

If all of the plugins for your application follow the same naming convention, you can use pkgutil.
iter_modules() to discover all of the top-level modules that match the naming convention. For example, Flask
uses the naming convention flask_{plugin_name}. If you wanted to automatically discover all of the Flask
plugins installed:

import importlib
import pkgutil

flask_plugins = {
name: importlib.import_module(name)
for finder, name, ispkg
in pkgutil.iter_modules()
if [Link]('flask_')
}

If you had both the Flask-SQLAlchemy and Flask-Talisman plugins installed then flask_plugins would be:

{
'flask_sqlachemy': <module: 'flask_sqlalchemy'>,
'flask_talisman': <module: 'flask_talisman'>,
}

Using naming convention for plugins also allows you to query the Python Package Index’s simple API for all packages
that conform to your naming convention.

3.14.2 Using namespace packages

Namespace packages can be used to provide a convention for where to place plugins and also provides a way
to perform discovery. For example, if you make the sub-package [Link] a namespace package then
other distributions can provide modules and packages to that namespace. Once installed, you can use pkgutil.
iter_modules() to discover all modules and packages installed under that namespace:

import importlib
import pkgutil

import [Link]

def iter_namespace(ns_pkg):
# Specifying the second argument (prefix) to iter_modules makes the
# returned name an absolute name instead of a relative one. This allows
# import_module to work without having to do additional modification to
# the name.
return pkgutil.iter_modules(ns_pkg.__path__, ns_pkg.__name__ + ".")

myapp_plugins = {
name: importlib.import_module(name)
for finder, name, ispkg
(continues on next page)

70 Chapter 3. Guides
 Python Packaging User Guide

(continued from previous page)

in iter_namespace([Link])
}

Specifying [Link].__path__ to iter_modules() causes it to only look for the modules directly
under that namespace. For example, if you have installed distributions that provide the modules [Link].a
and [Link].b then myapp_plugins in this case would be:

{
'a': <module: '[Link].a'>,
'b': <module: '[Link].b'>,
}

This sample uses a sub-package as the namespace package ([Link]), but it’s also possible to use a top-level
package for this purpose (such as myapp_plugins). How to pick the namespace to use is a matter of preference,
but it’s not recommended to make your project’s main top-level package ( myapp in this case) a namespace package
for the purpose of plugins, as one bad plugin could cause the entire namespace to break which would in turn make
your project unimportable. For the “namespace sub-package” approach to work, the plugin packages must omit
the __init__.py for your top-level package directory (myapp in this case) and include the namespace-package
style __init__.py in the namespace sub-package directory (myapp/plugins). This also means that plugins
will need to explicitly pass a list of packages to setup()’s packages argument instead of using setuptools.
find_packages().

Warning: Namespace packages are a complex feature and there are several different ways to create them. It’s
highly recommended to read the Packaging namespace packages documentation and clearly document which
approach is preferred for plugins to your project.

3.14.3 Using package metadata

Setuptools provides special support for plugins. By providing the entry_points argument to setup() in
[Link] plugins can register themselves for discovery.
For example if you have a package named myapp-plugin-a and it includes in its [Link]:

setup(
...
entry_points={'[Link]': 'a = myapp_plugin_a'},
...
)

Then you can discover and load all of the registered entry points by using pkg_resources.
iter_entry_points():

import pkg_resources

plugins = {
entry_point.name: entry_point.load()
for entry_point
in pkg_resources.iter_entry_points('[Link]')
}

In this example, plugins would be :

3.14. Creating and discovering plugins 71

Python Packaging User Guide

{
'a': <module: 'myapp_plugin_a'>,
}

Note: The entry_point specification in [Link] is fairly flexible and has a lot of options. It’s recommended
to read over the entire section on entry points.

3.15 Analyzing PyPI package downloads

This section covers how to use the PyPI package dataset to learn more about downloads of a package (or packages)
hosted on PyPI. For example, you can use it to discover the distribution of Python versions used to download a package.

Contents

• Background
• Setting up
• Useful queries
– Counting package downloads
– Package downloads over time
– More queries
• Additional tools
– pypinfo
– Other libraries

3.15.1 Background

PyPI does not display download statistics because they are difficult to collect and display accurately. Reasons for this
are included in the announcement email:
There are numerous reasons for [download counts] removal/deprecation some of which are:
• Technically hard to make work with the new CDN
– The CDN is being donated to the PSF, and the donated tier does not offer any form of log access
– The work around for not having log access would greatly reduce the utility of the CDN
• Highly inaccurate
– A number of things prevent the download counts from being inaccurate, some of which
include:

* pip download cache

* Internal or unofficial mirrors
* Packages not hosted on PyPI (for comparisons sake)

72 Chapter 3. Guides
 Python Packaging User Guide

* Mirrors or unofficial grab scripts causing inflated counts (Last I looked 25% of the down-
loads were from a known mirroring script).
• Not particularly useful
– Just because a project has been downloaded a lot doesn’t mean it’s good
– Similarly just because a project hasn’t been downloaded a lot doesn’t mean it’s bad
In short because it’s value is low for various reasons, and the tradeoffs required to make it work are high
It has been not an effective use of resources.
As an alternative, the Linehaul project streams download logs to Google BigQuery1 . Linehaul writes an entry in a
[Link] table for each download. The table contains information about what file
was downloaded and how it was downloaded. Some useful columns from the table schema include:

Column Description Examples

[Link] Project name pipenv, nose
[Link] Package version 0.1.6, 1.4.2
[Link] Installer pip, bandersnatch
[Link] Python version 2.7.12, 3.6.4

3.15.2 Setting up

In order to use Google BigQuery to query the PyPI package dataset, you’ll need a Google account and to enable
the BigQuery API on a Google Cloud Platform project. You can run the up to 1TB of queries per month using the
BigQuery free tier without a credit card
• Navigate to the BigQuery web UI.
• Create a new project.
• Enable the BigQuery API.
For more detailed instructions on how to get started with BigQuery, check out the BigQuery quickstart guide.

3.15.3 Useful queries

Run queries in the BigQuery web UI by clicking the “Compose query” button.
Note that the rows are stored in separate tables for each day, which helps limit the cost of queries. These exam-
ple queries analyze downloads from recent history by using wildcard tables to select all tables and then filter by
_TABLE_SUFFIX.

Counting package downloads

The following query counts the total number of downloads for the project “pytest”.

#standardSQL
SELECT COUNT(*) AS num_downloads
FROM `[Link]*`
WHERE [Link] = 'pytest'
-- Only query the last 30 days of history
AND _TABLE_SUFFIX
(continues on next page)
1 PyPI BigQuery dataset announcement email

3.15. Analyzing PyPI package downloads 73

Python Packaging User Guide

(continued from previous page)

BETWEEN FORMAT_DATE(
'%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY))
AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())

num_downloads
2117807

To only count downloads from pip, filter on the [Link] column.

#standardSQL
SELECT COUNT(*) AS num_downloads
FROM `[Link]*`
WHERE [Link] = 'pytest'
AND [Link] = 'pip'
-- Only query the last 30 days of history
AND _TABLE_SUFFIX
BETWEEN FORMAT_DATE(
'%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY))
AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())

num_downloads
1829322

Package downloads over time

To group by monthly downloads, use the _TABLE_SUFFIX pseudo-column. Also use the pseudo-column to limit the
tables queried and the corresponding costs.

#standardSQL
SELECT
COUNT(*) AS num_downloads,
SUBSTR(_TABLE_SUFFIX, 1, 6) AS `month`
FROM `[Link]*`
WHERE
[Link] = 'pytest'
-- Only query the last 6 months of history
AND _TABLE_SUFFIX
BETWEEN FORMAT_DATE(
'%Y%m01', DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH))
AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
GROUP BY `month`
ORDER BY `month` DESC

num_downloads month
1956741 201801
2344692 201712
1730398 201711
2047310 201710
1744443 201709
1916952 201708

74 Chapter 3. Guides
 Python Packaging User Guide

More queries

• Data driven decisions using PyPI download statistics

• PyPI queries gist
• Python versions over time
• Non-Windows downloads, grouped by platform

3.15.4 Additional tools

You can also access the PyPI package dataset programmatically via the BigQuery API.

pypinfo

pypinfo is a command-line tool which provides access to the dataset and can generate several useful queries. For
example, you can query the total number of download for a package with the command pypinfo package_name.

$ pypinfo requests
Served from cache: False
Data processed: 6.87 GiB
Data billed: 6.87 GiB
Estimated cost: $0.04

| download_count |
| -------------- |
| 9,316,415 |

Install pypinfo using pip.

pip install pypinfo

Other libraries

• google-cloud-bigquery is the official client library to access the BigQuery API.

• pandas-gbq allows for accessing query results via Pandas.

3.16 Package index mirrors and caches

Page Status Incomplete

Last Reviewed 2014-12-24

Contents

• Caching with pip

• Caching with devpi
• Complete mirror with bandersnatch

3.16. Package index mirrors and caches 75

Python Packaging User Guide

Mirroring or caching of PyPI can be used to speed up local package installation, allow offline work, handle corporate
firewalls or just plain Internet flakiness.
Three options are available in this area:
1. pip provides local caching options,
2. devpi provides higher-level caching option, potentially shared amongst many users or machines, and
3. bandersnatch provides a local complete mirror of all PyPI packages.

3.16.1 Caching with pip

pip provides a number of facilities for speeding up installation by using local cached copies of packages:
1. Fast & local installs by downloading all the requirements for a project and then pointing pip at those downloaded
files instead of going to PyPI.
2. A variation on the above which pre-builds the installation files for the requirements using pip wheel:

$ pip wheel --wheel-dir=/tmp/wheelhouse SomeProject

$ pip install --no-index --find-links=/tmp/wheelhouse SomeProject

3.16.2 Caching with devpi

devpi is a caching proxy server which you run on your laptop, or some other machine you know will always be
available to you. See the devpi documentation for getting started.

3.16.3 Complete mirror with bandersnatch

bandersnatch will set up a complete local mirror of all PyPI packages (externally-hosted packages are not mirrored).
See the bandersnatch documentation for getting that going.
A benefit of devpi is that it will create a mirror which includes packages that are external to PyPI, unlike bandersnatch
which will only cache packages hosted on PyPI.

3.17 Hosting your own simple repository

If you wish to host your own simple repository1 , you can either use a software package like devpi or you can use simply
create the proper directory structure and use any web server that can serve static files and generate an autoindex.
In either case, since you’ll be hosting a repository that is likely not in your user’s default repositories, you should
instruct them in your project’s description to configure their installer appropriately. For example with pip:

pip install --extra-index-url [Link] foobar

In addition, it is highly recommended that you serve your repository with valid HTTPS. At this time, the security of
your user’s installations depends on all repositories using a valid HTTPS setup.
1 For complete documentation of the simple repository protocol, see PEP 503.

76 Chapter 3. Guides
 Python Packaging User Guide

3.17.1 “Manual” repository

The directory layout is fairly simple, within a root directory you need to create a directory for each project. This
directory should be the normalized name (as defined by PEP 503) of the project. Within each of these directories
simply place each of the downloadable files. If you have the projects “Foo” (with the versions 1.0 and 2.0) and “bar”
(with the version 0.1) You should end up with a structure that looks like:

.
bar
[Link]
foo
[Link]
[Link]

Once you have this layout, simply configure your webserver to serve the root directory with autoindex enabled. For
an example using the built in Web server in Twisted, you would simply run twistd -n web --path . and then
instruct users to add the URL to their installer’s configuration.

3.18 Migrating to [Link]

[Link] is the new, rewritten version of PyPI that has replaced the legacy PyPI code base. It is the default version
of PyPI that people are expected to use. These are the tools and processes that people will need to interact with
[Link].

3.18.1 Publishing releases

[Link] is the default upload platform as of September 2016.

Uploads through [Link] were switched off on July 3, 2017. As of April 13th, 2018,‘‘[Link]‘‘ is the
URL for PyPI.
The recommended way to migrate to [Link] for uploading is to ensure that you are using a new enough version of
your upload tool.
The default upload settings switched to [Link] in the following versions:
• twine 1.8.0
• setuptools 27.0.0
• Python 2.7.13 (distutils update)
• Python 3.4.6 (distutils update)
• Python 3.5.3 (distutils update)
• Python 3.6.0 (distutils update)
In addition to ensuring you’re on a new enough version of the tool for the tool’s default to have switched, you must
also make sure that you have not configured the tool to override its default upload URL. Typically this is configured
in a file located at $HOME/.pypirc. If you see a file like:

3.18. Migrating to [Link] 77

Python Packaging User Guide

[distutils]
index-servers =
pypi

[pypi]
repository:[Link]
username:yourusername
password:yourpassword

Then simply delete the line starting with repository and you will use your upload tool’s default URL.
If for some reason you’re unable to upgrade the version of your tool to a version that defaults to using [Link],
then you may edit $HOME/.pypirc and include the repository: line, but use the value [Link]
[Link]/legacy/ instead:

[distutils]
index-servers =
pypi

[pypi]
repository: [Link]
username: your username
password: your password

(legacy in this URL refers to the fact that this is the new server implementation’s emulation of the legacy server
implementation’s upload API.)

3.18.2 Registering package names & metadata

Explicit pre-registration of package names with the [Link] register command prior to the first upload is no
longer required, and is not currently supported by the legacy upload API emulation on [Link].
As a result, attempting explicit registration after switching to using [Link] for uploads will give the following error
message:

Server response (410): This API is no longer supported, instead simply upload the
˓→file.

The solution is to skip the registration step, and proceed directly to uploading artifacts.

3.18.3 Using TestPyPI

Legacy TestPyPI ([Link]) is no longer available; use [Link] instead. If you use TestPyPI, you must
update your $HOME/.pypirc to handle TestPyPI’s new location, by replacing [Link]
org/pypi with [Link] for example:

[distutils]
index-servers=
pypi
testpypi

[testpypi]
repository: [Link]
username: your testpypi username
password: your testpypi password

78 Chapter 3. Guides
 Python Packaging User Guide

3.18.4 Registering new user accounts

In order to help mitigate spam attacks against PyPI, new user registration through [Link] was switched
off on February 20, 2018. New user registrations at [Link] are open.

3.18.5 Browsing packages

While [Link] is may still be used in links from other PyPA documentation, etc, the default interface
for browsing packages is [Link]. The domain [Link] now redirects to [Link], and may be disabled
sometime in the future.

3.18.6 Downloading packages

[Link] is the default host for downloading packages.

3.18.7 Managing published packages and releases

[Link] provides a fully functional interface for logged in users to manage their published packages and releases.

3.19 Using TestPyPI

TestPyPI is a separate instance of the Python Package Index (PyPI) that allows you to try out the distribution tools and
process without worrying about affecting the real index. TestPyPI is hosted at [Link]

3.19.1 Registering your account

Because TestPyPI has a separate database from the live PyPI, you’ll need a separate user account for specifically for
TestPyPI. Go to [Link] to register your account.

Note: The database for TestPyPI may be periodically pruned, so it is not unusual for user accounts to be deleted.

3.19.2 Using TestPyPI with Twine

You can upload your distributions to TestPyPI using twine by passing in the --repository-url flag

$ twine upload --repository-url [Link] dist/*

The legacy in the URL above refers to the legacy API for PyPI, which may change as the Warehouse project
progresses.
You can see if your package has successfully uploaded by navigating to the URL [Link]
project/<sampleproject> where sampleproject is the name of your project that you uploaded. It may
take a minute or two for your project to appear on the site.

3.19. Using TestPyPI 79

Python Packaging User Guide

3.19.3 Using TestPyPI with pip

You can tell pip to download packages from TestPyPI instead of PyPI by specifying the --index-url flag

$ pip install --index-url [Link] your-package

If you want to allow pip to also pull other packages from PyPI you can specify --extra-index-url to point to
PyPI. This is useful when the package you’re testing has dependencies:

pip install --index-url [Link] --extra-index-url [Link]

˓→org/simple your-package

3.19.4 Setting up TestPyPI in pypirc

If you want to avoid entering the TestPyPI url and your username you can configure TestPyPI in your $HOME/.
pypirc.
Create or modify $HOME/.pypirc with the following:

[distutils]
index-servers=
pypi
testpypi

[testpypi]
repository: [Link]
username: your testpypi username

Warning: Do not store passwords in the pypirc file. Storing passwords in plain text is never a good idea.

You can then tell twine to upload to TestPyPI by specifying the --repository flag:

$ twine upload --repository testpypi dist/*

3.20 Making a PyPI-friendly README

README files can help your users understand your project and can be used to set your project’s description on PyPI.
This guide helps you create a README in a PyPI-friendly format and include your README in your package so it
appears on PyPI.

3.20.1 Creating a README file

README files for Python projects are often named README, [Link], [Link], or [Link].
For your README to display properly on PyPI, choose a markup language supported by PyPI. Formats supported by
PyPI’s README renderer are:
• plain text
• reStructuredText (without Sphinx extensions)
• Markdown (GitHub Flavored Markdown by default, or CommonMark)

80 Chapter 3. Guides
 Python Packaging User Guide

It’s customary to save your README file in the root of your project, in the same directory as your [Link] file.

3.20.2 Including your README in your package’s metadata

To include your README’s contents as your package description, set your project’s Description and
Description-Content-Type metadata, typically in your project’s [Link] file.
See also:
• Description
• Description-Content-Type
For example, to set these values in a package’s [Link] file, use setup()’s long_description and
long_description_content_type.
Set the value of long_description to the contents (not the path) of the README file itself. Set the
long_description_content_type to an accepted Content-Type-style value for your README file’s
markup, such as text/plain, text/x-rst (for reStructuredText), or text/markdown.

Note: If you’re using GitHub-flavored Markdown to write a project’s description, ensure you upgrade the following
tools:

python3 -m pip install --user --upgrade setuptools wheel twine

The minimum required versions of the respective tools are:

• setuptools >= 38.6.0
• wheel >= 0.31.0
• twine >= 1.11.0
It’s recommended that you use twine to upload the project’s distribution packages:

twine upload dist/*

For example, see this [Link] file, which reads the contents of [Link] as long_description and iden-
tifies the markup as GitHub-flavored Markdown:

from setuptools import setup

# read the contents of your README file

from os import path
this_directory = [Link]([Link](__file__))
with open([Link](this_directory, '[Link]'), encoding='utf-8') as f:
long_description = [Link]()

setup(
name='an_example_package',
# other arguments omitted
long_description=long_description,
long_description_content_type='text/markdown'
)

3.20. Making a PyPI-friendly README 81

Python Packaging User Guide

3.20.3 Validating reStructuredText markup

If your README is written in reStructuredText, any invalid markup will prevent it from rendering, causing PyPI to
instead just show the README’s raw source.
Note that Sphinx extensions used in docstrings, such as directives and roles (e.g., “:py:func:`getattr`”
or “:ref:`my-reference-label`”), are not allowed here and will result in error messages like “Error:
Unknown interpreted text role "py:func".”.
You can check your README for markup errors before uploading as follows:
1. Install the latest version of twine; version 1.12.0 or higher is required:

pip install --upgrade twine

2. Build the sdist and wheel for your project as described under Packaging your project.
3. Run twine check on the sdist and wheel:

twine check dist/*

This command will report any problems rendering your README. If your markup renders fine, the command
will output Checking distribution FILENAME: Passed.

82 Chapter 3. Guides
 CHAPTER 4

Discussions

Discussions are focused on providing comprehensive information about a specific topic. If you’re just trying to get
stuff done, see Guides.

4.1 Deploying Python applications

Page Status Incomplete

Last Reviewed 2014-11-11

Contents

• Overview
– Supporting multiple hardware platforms
• OS packaging & installers
– Windows

* Pynsist
• Application bundles
• Configuration management

4.1.1 Overview

Supporting multiple hardware platforms

83
Python Packaging User Guide

FIXME

Meaning: x86, x64, ARM, others?

For Python-only distributions, it *should* be straightforward to deploy on all

platforms where Python can run.

For distributions with binary extensions, deployment is major headache. Not only
must the extensions be built on all the combinations of operating system and
hardware platform, but they must also be tested, preferably on continuous
integration platforms. The issues are similar to the "multiple Python
versions" section above, not sure whether this should be a separate section.
Even on Windows x64, both the 32 bit and 64 bit versions of Python enjoy
significant usage.

4.1.2 OS packaging & installers

FIXME

- Building rpm/debs for projects

- Building rpms/debs for whole virtualenvs
- Building macOS installers for Python projects
- Building Android APKs with Kivy+P4A or P4A & Buildozer

Windows

FIXME

- Building Windows installers for Python projects

Pynsist

Pynsist is a tool that bundles Python programs together with the Python-interpreter into a single installer based on
NSIS. In most cases, packaging only requires the user to choose a version of the Python-interpreter and declare the
dependencies of the program. The tool downloads the specified Python-interpreter for Windows and packages it with
all the dependencies in a single Windows-executable installer.
The installer installs or updates the Python-interpreter on the users system, which can be used independently of the
packaged program. The program itself, can be started from a shortcut, that the installer places in the start-menu.
Uninstalling the program leaves the Python installation of the user intact.
A big advantage of pynsist is that the Windows packages can be built on Linux. There are several examples for
different kinds of programs (console, GUI) in the documentation. The tool is released under the MIT-licence.

4.1.3 Application bundles

FIXME

- py2exe/py2app/PEX
- wheels kinda/sorta

84 Chapter 4. Discussions
 Python Packaging User Guide

4.1.4 Configuration management

FIXME

puppet
salt
chef
ansible
fabric

4.2 pip vs easy_install

easy_install was released in 2004, as part of setuptools. It was notable at the time for installing packages from PyPI
using requirement specifiers, and automatically installing dependencies.
pip came later in 2008, as alternative to easy_install, although still largely built on top of setuptools components. It
was notable at the time for not installing packages as Eggs or from Eggs (but rather simply as ‘flat’ packages from
sdists), and introducing the idea of Requirements Files, which gave users the power to easily replicate environments.
Here’s a breakdown of the important differences between pip and easy_install now:

pip easy_install
Installs from Wheels Yes No
Uninstall Packages Yes (pip uninstall) No
Dependency Overrides Yes (Requirements Files) No
List Installed Packages Yes (pip list and pip freeze) No
Yes No
PEP 438 Support
Installation format ‘Flat’ packages with egg-info metadata. Encapsulated Egg format
[Link] modification No Yes
Installs from Eggs No Yes
pylauncher support No Yes1
Multi-version installs No Yes
Exclude scripts during install No Yes
per project index Only in virtualenv Yes, via [Link]

4.3 install_requires vs requirements files

Contents

• install_requires
• Requirements files

1 [Link]

4.2. pip vs easy_install 85

Python Packaging User Guide

4.3.1 install_requires

install_requires is a setuptools [Link] keyword that should be used to specify what a project minimally
needs to run correctly. When the project is installed by pip, this is the specification that is used to install its dependen-
cies.
For example, if the project requires A and B, your install_requires would be like so:

install_requires=[
'A',
'B'
]

Additionally, it’s best practice to indicate any known lower or upper bounds.
For example, it may be known, that your project requires at least v1 of ‘A’, and v2 of ‘B’, so it would be like so:

install_requires=[
'A>=1',
'B>=2'
]

It may also be known that project A follows semantic versioning, and that v2 of ‘A’ will indicate a break in compati-
bility, so it makes sense to not allow v2:

install_requires=[
'A>=1,<2',
'B>=2'
]

It is not considered best practice to use install_requires to pin dependencies to specific versions, or to specify
sub-dependencies (i.e. dependencies of your dependencies). This is overly-restrictive, and prevents the user from
gaining the benefit of dependency upgrades.
Lastly, it’s important to understand that install_requires is a listing of “Abstract” requirements, i.e just names
and version restrictions that don’t determine where the dependencies will be fulfilled from (i.e. from what index or
source). The where (i.e. how they are to be made “Concrete”) is to be determined at install time using pip options.1

4.3.2 Requirements files

Requirements Files described most simply, are just a list of pip install arguments placed into a file.
Whereas install_requires defines the dependencies for a single project, Requirements Files are often used to
define the requirements for a complete Python environment.
Whereas install_requires requirements are minimal, requirements files often contain an exhaustive listing of
pinned versions for the purpose of achieving repeatable installations of a complete environment.
Whereas install_requires requirements are “Abstract”, i.e. not associated with any particular index, require-
ments files often contain pip options like --index-url or --find-links to make requirements “Concrete”, i.e.
associated with a particular index or directory of packages.1
Whereas install_requires metadata is automatically analyzed by pip during an install, requirements files are
not, and only are used when a user specifically installs them using pip install -r.

1 For more on “Abstract” vs “Concrete” requirements, see [Link]

86 Chapter 4. Discussions
 Python Packaging User Guide

4.4 Wheel vs Egg

Wheel and Egg are both packaging formats that aim to support the use case of needing an install artifact that doesn’t
require building or compilation, which can be costly in testing and production workflows.
The Egg format was introduced by setuptools in 2004, whereas the Wheel format was introduced by PEP 427 in 2012.
Wheel is currently considered the standard for built and binary packaging for Python.
Here’s a breakdown of the important differences between Wheel and Egg.
• Wheel has an official PEP. Egg did not.
• Wheel is a distribution format, i.e a packaging format.1 Egg was both a distribution format and a runtime
installation format (if left zipped), and was designed to be importable.
• Wheel archives do not include .pyc files. Therefore, when the distribution only contains Python files (i.e. no
compiled extensions), and is compatible with Python 2 and 3, it’s possible for a wheel to be “universal”, similar
to an sdist.
• Wheel uses PEP376-compliant .dist-info directories. Egg used .egg-info.
• Wheel has a richer file naming convention. A single wheel archive can indicate its compatibility with a number
of Python language versions and implementations, ABIs, and system architectures.
• Wheel is versioned. Every wheel file contains the version of the wheel specification and the implementation that
packaged it.
• Wheel is internally organized by sysconfig path type, therefore making it easier to convert to other formats.

1 Circumstantially, in some cases, wheels can be used as an importable runtime format, although this is not officially supported at this time.

4.4. Wheel vs Egg 87

Python Packaging User Guide

88 Chapter 4. Discussions
 CHAPTER 5

PyPA specifications

This is a list of currently active interoperability specifications maintained by the Python Packaging Authority. The
process for updating these standards, and for proposing new ones, is documented on [Link].

5.1 Package Distribution Metadata

5.1.1 Core metadata specifications

The current core metadata file format, version 2.1, is specified in PEP 566. It defines the following specification as
the canonical source for the core metadata file format.
Fields defined in the following specification should be considered valid, complete and not subject to change. The
required fields are:
• Metadata-Version
• Name
• Version
All the other fields are optional.

Note: Interpreting old metadata: In PEP 566, the version specifier field format specification was relaxed to accept the
syntax used by popular publishing tools (namely to remove the requirement that version specifiers must be surrounded
by parentheses). Metadata consumers may want to use the more relaxed formatting rules even for metadata files that
are nominally less than version 2.1.

Contents

• Metadata-Version

89
Python Packaging User Guide

• Name
• Version
• Platform (multiple use)
• Supported-Platform (multiple use)
• Summary
• Description
• Description-Content-Type
• Keywords
• Home-page
• Download-URL
• Author
• Author-email
• Maintainer
• Maintainer-email
• License
• Classifier (multiple use)
• Requires-Dist (multiple use)
• Requires-Python
• Requires-External (multiple use)
• Project-URL (multiple-use)
• Provides-Extra (multiple use)
• Rarely Used Fields
– Provides-Dist (multiple use)
– Obsoletes-Dist (multiple use)

Metadata-Version

New in version 1.0.

Version of the file format; legal values are “1.0”, “1.1”, “1.2” and “2.1”.
Automated tools consuming metadata SHOULD warn if metadata_version is greater than the highest version
they support, and MUST fail if metadata_version has a greater major version than the highest version they
support (as described in PEP 440, the major version is the value before the first dot).
For broader compatibility, build tools MAY choose to produce distribution metadata using the lowest metadata version
that includes all of the needed fields.
Example:

Metadata-Version: 2.1

90 Chapter 5. PyPA specifications

 Python Packaging User Guide

Name

New in version 1.0.

Changed in version 2.1: Added additional restrictions on format from PEP 508
The name of the distribution. The name field is the primary identifier for a distribution. A valid name consists only of
ASCII letters and numbers, period, underscore and hyphen. It must start and end with a letter or number. Distribution
names are limited to those which match the following regex (run with [Link]):

^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$

Example:

Name: BeagleVote

Version

New in version 1.0.

A string containing the distribution’s version number. This field must be in the format specified in PEP 440.
Example:

Version: 1.0a2

Platform (multiple use)

New in version 1.0.

A Platform specification describing an operating system supported by the distribution which is not listed in the “Op-
erating System” Trove classifiers. See “Classifier” below.
Examples:

Platform: ObscureUnix
Platform: RareDOS

Supported-Platform (multiple use)

New in version 1.1.

Binary distributions containing a PKG-INFO file will use the Supported-Platform field in their metadata to specify the
OS and CPU for which the binary distribution was compiled. The semantics of the Supported-Platform field are not
specified in this PEP.
Example:

Supported-Platform: RedHat 7.2

Supported-Platform: i386-win32-2791

5.1. Package Distribution Metadata 91

Python Packaging User Guide

Summary

New in version 1.0.

A one-line summary of what the distribution does.
Example:

Summary: A module for collecting votes from beagles.

Description

New in version 1.0.

Changed in version 2.1: This field may be specified in the message body instead.
A longer description of the distribution that can run to several paragraphs. Software that deals with metadata should not
assume any maximum size for this field, though people shouldn’t include their instruction manual as the description.
The contents of this field can be written using reStructuredText markup1 . For programs that work with the metadata,
supporting markup is optional; programs can also display the contents of the field as-is. This means that authors should
be conservative in the markup they use.
To support empty lines and lines with indentation with respect to the RFC 822 format, any CRLF character has to be
suffixed by 7 spaces followed by a pipe (“|”) char. As a result, the Description field is encoded into a folded field that
can be interpreted by RFC822 parser2 .
Example:

Description: This project provides powerful math functions

|For example, you can use `sum()` to sum numbers:
|
|Example::
|
| >>> sum(1, 2)
| 3
|

This encoding implies that any occurrences of a CRLF followed by 7 spaces and a pipe char have to be replaced by a
single CRLF when the field is unfolded using a RFC822 reader.
Alternatively, the distribution’s description may instead be provided in the message body (i.e., after a completely blank
line following the headers, with no indentation or other special formatting necessary).

Description-Content-Type

New in version 2.1.

A string stating the markup syntax (if any) used in the distribution’s description, so that tools can intelligently render
the description.
Historically, PyPI supported descriptions in plain text and reStructuredText (reST), and could render reST into HTML.
However, it is common for distribution authors to write the description in Markdown (RFC 7763) as many code hosting
sites render Markdown READMEs, and authors would reuse the file for the description. PyPI didn’t recognize the
format and so could not render the description correctly. This resulted in many packages on PyPI with poorly-rendered
descriptions when Markdown is left as plain text, or worse, was attempted to be rendered as reST. This field allows
1 reStructuredText markup: [Link]
2 RFC 822 Long Header Fields: [Link]

92 Chapter 5. PyPA specifications

 Python Packaging User Guide

the distribution author to specify the format of their description, opening up the possibility for PyPI and other tools to
be able to render Markdown and other formats.
The format of this field is the same as the Content-Type header in HTTP (i.e.: RFC 1341). Briefly, this means that
it has a type/subtype part and then it can optionally have a number of parameters:
Format:

Description-Content-Type: <type>/<subtype>; charset=<charset>[; <param_name>=<param

˓→value> ...]

The type/subtype part has only a few legal values:

• text/plain
• text/x-rst
• text/markdown
The charset parameter can be used to specify the character encoding of the description. The only legal value is
UTF-8. If omitted, it is assumed to be UTF-8.
Other parameters might be specific to the chosen subtype. For example, for the markdown subtype, there is an
optional variant parameter that allows specifying the variant of Markdown in use (defaults to GFM if not specified).
Currently, two variants are recognized:
• GFM for Github-flavored Markdown
• CommonMark for CommonMark
Example:

Description-Content-Type: text/plain; charset=UTF-8

Example:

Description-Content-Type: text/x-rst; charset=UTF-8

Example:

Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

Example:

Description-Content-Type: text/markdown

If a Description-Content-Type is not specified, then applications should attempt to render it as text/

x-rst; charset=UTF-8 and fall back to text/plain if it is not valid rst.
If a Description-Content-Type is an unrecognized value, then the assumed content type is text/plain
(Although PyPI will probably reject anything with an unrecognized value).
If the Description-Content-Type is text/markdown and variant is not specified or is set to an unrec-
ognized value, then the assumed variant is GFM.
So for the last example above, the charset defaults to UTF-8 and the variant defaults to GFM and thus it is
equivalent to the example before it.

Keywords

New in version 1.0.

5.1. Package Distribution Metadata 93

Python Packaging User Guide

A list of additional keywords to be used to assist searching for the distribution in a larger catalog.
Example:

Keywords: dog puppy voting election

Home-page

New in version 1.0.

A string containing the URL for the distribution’s home page.
Example:

Home-page: [Link]

Download-URL

New in version 1.1.

A string containing the URL from which this version of the distribution can be downloaded. (This means that the URL
can’t be something like “. . . /[Link]”, but instead must be “. . . /[Link]”.)

Author

New in version 1.0.

A string containing the author’s name at a minimum; additional contact information may be provided.
Example:

Author: C. Schultz, Universal Features Syndicate,

Los Angeles, CA <cschultz@[Link]>

Author-email

New in version 1.0.

A string containing the author’s e-mail address. It can contain a name and e-mail address in the legal forms for a
RFC-822 From: header.
Example:

Author-email: "C. Schultz" <cschultz@[Link]>

Per RFC-822, this field may contain multiple comma-separated e-mail addresses:

Author-email: cschultz@[Link], snoopy@[Link]

94 Chapter 5. PyPA specifications

 Python Packaging User Guide

Maintainer

New in version 1.2.

A string containing the maintainer’s name at a minimum; additional contact information may be provided.
Note that this field is intended for use when a project is being maintained by someone other than the original author:
it should be omitted if it is identical to Author.
Example:
Maintainer: C. Schultz, Universal Features Syndicate,
Los Angeles, CA <cschultz@[Link]>

Maintainer-email

New in version 1.2.

A string containing the maintainer’s e-mail address. It can contain a name and e-mail address in the legal forms for a
RFC-822 From: header.
Note that this field is intended for use when a project is being maintained by someone other than the original author:
it should be omitted if it is identical to Author-email.
Example:
Maintainer-email: "C. Schultz" <cschultz@[Link]>

Per RFC-822, this field may contain multiple comma-separated e-mail addresses:
Maintainer-email: cschultz@[Link], snoopy@[Link]

License

New in version 1.0.

Text indicating the license covering the distribution where the license is not a selection from the “License” Trove
classifiers. See “Classifier” below. This field may also be used to specify a particular version of a license which is
named via the Classifier field, or to indicate a variation or exception to such a license.
Examples:
License: This software may only be obtained by sending the
author a postcard, and then the user promises not
to redistribute it.

License: GPL version 3, excluding DRM provisions

Classifier (multiple use)

New in version 1.1.

Each entry is a string giving a single classification value for the distribution. Classifiers are described in PEP 301, and
the Python Package Index publishes a dynamic list of currently defined classifiers.
This field may be followed by an environment marker after a semicolon.
Examples:

5.1. Package Distribution Metadata 95

Python Packaging User Guide

Classifier: Development Status :: 4 - Beta

Classifier: Environment :: Console (Text Based)

Requires-Dist (multiple use)

New in version 1.2.

Changed in version 2.1: The field format specification was relaxed to accept the syntax used by popular publishing
tools.
Each entry contains a string naming some other distutils project required by this distribution.
The format of a requirement string contains from one to four parts:
• A project name, in the same format as the Name: field. The only mandatory part.
• A comma-separated list of ‘extra’ names. These are defined by the required project, referring to specific features
which may need extra dependencies.
• A version specifier. Tools parsing the format should accept optional parentheses around this, but tools generating
it should not use parentheses.
• An environment marker after a semicolon. This means that the requirement is only needed in the specified
conditions.
See PEP 508 for full details of the allowed format.
The project names should correspond to names as found on the Python Package Index.
Version specifiers must follow the rules described in Version specifiers.
Examples:

Requires-Dist: pkginfo
Requires-Dist: PasteDeploy
Requires-Dist: [Link] (>3.5.0)
Requires-Dist: pywin32 >1.0; sys_platform == 'win32'

Requires-Python

New in version 1.2.

This field specifies the Python version(s) that the distribution is guaranteed to be compatible with. Installation tools
may look at this when picking which version of a project to install.
The value must be in the format specified in Version specifiers.
This field may be followed by an environment marker after a semicolon.
Examples:

Requires-Python: >=3
Requires-Python: >2.6,!=3.0.*,!=3.1.*
Requires-Python: ~=2.6
Requires-Python: >=3; sys_platform == 'win32'

96 Chapter 5. PyPA specifications

 Python Packaging User Guide

Requires-External (multiple use)

New in version 1.2.

Changed in version 2.1: The field format specification was relaxed to accept the syntax used by popular publishing
tools.
Each entry contains a string describing some dependency in the system that the distribution is to be used. This field
is intended to serve as a hint to downstream project maintainers, and has no semantics which are meaningful to the
distutils distribution.
The format of a requirement string is a name of an external dependency, optionally followed by a version declaration
within parentheses.
This field may be followed by an environment marker after a semicolon.
Because they refer to non-Python software releases, version numbers for this field are not required to conform to the
format specified in PEP 440: they should correspond to the version scheme used by the external dependency.
Notice that there is no particular rule on the strings to be used.
Examples:

Requires-External: C
Requires-External: libpng (>=1.5)
Requires-External: make; sys_platform != "win32"

Project-URL (multiple-use)

New in version 1.2.

A string containing a browsable URL for the project and a label for it, separated by a comma.
Example:

Bug Tracker, [Link]

The label is a free text limited to 32 signs.

Provides-Extra (multiple use)

New in version 2.1.

A string containing the name of an optional feature. Must be a valid Python identifier. May be used to make a
dependency conditional on whether the optional feature has been requested.
Example:

Provides-Extra: pdf
Requires-Dist: reportlab; extra == 'pdf'

A second distribution requires an optional dependency by placing it inside square brackets, and can request multiple
features by separating them with a comma (,). The requirements are evaluated for each requested feature and added to
the set of requirements for the distribution.
Example:

Requires-Dist: beaglevote[pdf]
Requires-Dist: libexample[test, doc]

5.1. Package Distribution Metadata 97

Python Packaging User Guide

Two feature names test and doc are reserved to mark dependencies that are needed for running automated tests and
generating documentation, respectively.
It is legal to specify Provides-Extra: without referencing it in any Requires-Dist:.

Rarely Used Fields

The fields in this section are currently rarely used, as their design was inspired by comparable mechanisms in Linux
package management systems, and it isn’t at all clear how tools should interpret them in the context of an open index
server such as PyPI.
As a result, popular installation tools ignore them completely, which in turn means there is little incentive for package
publishers to set them appropriately. However, they’re retained in the metadata specification, as they’re still potentially
useful for informational purposes, and can also be used for their originally intended purpose in combination with a
curated package repository.

Provides-Dist (multiple use)

New in version 1.2.

Changed in version 2.1: The field format specification was relaxed to accept the syntax used by popular publishing
tools.
Each entry contains a string naming a Distutils project which is contained within this distribution. This field must
include the project identified in the Name field, followed by the version : Name (Version).
A distribution may provide additional names, e.g. to indicate that multiple projects have been bundled together. For
instance, source distributions of the ZODB project have historically included the transaction project, which is
now available as a separate distribution. Installing such a source distribution satisfies requirements for both ZODB and
transaction.
A distribution may also provide a “virtual” project name, which does not correspond to any separately-distributed
project: such a name might be used to indicate an abstract capability which could be supplied by one of multiple
projects. E.g., multiple projects might supply RDBMS bindings for use by a given ORM: each project might declare
that it provides ORM-bindings, allowing other projects to depend only on having at most one of them installed.
A version declaration may be supplied and must follow the rules described in Version specifiers. The distribution’s
version number will be implied if none is specified.
This field may be followed by an environment marker after a semicolon.
Examples:

Provides-Dist: OtherProject
Provides-Dist: AnotherProject (3.4)
Provides-Dist: virtual_package; python_version >= "3.4"

Obsoletes-Dist (multiple use)

New in version 1.2.

Changed in version 2.1: The field format specification was relaxed to accept the syntax used by popular publishing
tools.
Each entry contains a string describing a distutils project’s distribution which this distribution renders obsolete, mean-
ing that the two projects should not be installed at the same time.

98 Chapter 5. PyPA specifications

 Python Packaging User Guide

Version declarations can be supplied. Version numbers must be in the format specified in Version specifiers.
This field may be followed by an environment marker after a semicolon.
The most common use of this field will be in case a project name changes, e.g. Gorgon 2.3 gets subsumed into Torqued
Python 1.0. When you install Torqued Python, the Gorgon distribution should be removed.
Examples:

Obsoletes-Dist: Gorgon
Obsoletes-Dist: OtherProject (<3.0)
Obsoletes-Dist: Foo; os_name == "posix"

5.1.2 Version specifiers

Version numbering requirements and the semantics for specifying comparisons between versions are defined in PEP
440.
The version specifiers section in this PEP supersedes the version specifiers section in PEP 345.

5.1.3 Dependency specifiers

The dependency specifier format used to declare a dependency on another component is defined in PEP 508.
The environment markers section in this PEP supersedes the environment markers section in PEP 345.

5.1.4 Declaring build system dependencies

[Link] is a build system independent file format defined in PEP 518 that projects may provide in order to
declare any Python level dependencies that must be installed in order to run the project’s build system successfully.

5.1.5 Distribution formats

Source distribution format

The accepted style of source distribution format based on [Link], defined in PEP 518 and adopted by
PEP 517 has not been implemented yet.
There is also the legacy source distribution format, implicitly defined by the behaviour of distutils module in the
standard library, when executing [Link] sdist.

Binary distribution format

The binary distribution format (wheel) is defined in PEP 427.

5.1.6 Platform compatibility tags

The platform compatibility tagging model used for wheel distribution is defined in PEP 425.

5.1. Package Distribution Metadata 99

Python Packaging User Guide

Manylinux tags

The scheme defined in PEP 425 is insufficient for public distribution of Linux wheel files (and *nix wheel files in gen-
eral), so the manylinux platform tags were defined, to allow providing wheels for many common Linux distributions.
See PEP 513 for more about how this works.
• manylinux1 is defined in PEP 513, for x86_64 and i686 architectures.
• manylinux2010 is defined in PEP 571, for x86_64 and i686 architectures. It is based on a platform from
2010, whereas manylinux1 is based on a platform from 2007. This means that manylinux2010 packages
are easier to create, but not compatible with some older systems where manylinux1 packages would work.
manylinux2010 is not yet widely recognised by install tools.

5.1.7 Recording installed distributions

The format used to record installed packages and their contents is defined in PEP 376.
Note that only the dist-info directory and the RECORD file format from that PEP are currently implemented in the
default packaging toolchain.

5.1.8 Entry points specification

Entry points are a mechanism for an installed distribution to advertise components it provides to be discovered and
used by other code. For example:
• Distributions can specify console_scripts entry points, each referring to a function. When pip (or another
console_scripts aware installer) installs the distribution, it will create a command-line wrapper for each entry
point.
• Applications can use entry points to load plugins; e.g. Pygments (a syntax highlighting tool) can use additional
lexers and styles from separately installed packages. For more about this, see Creating and discovering plugins.
The entry point file format was originally developed to allow packages built with setuptools to provide integration
point metadata that would be read at runtime with pkg_resources. It is now defined as a PyPA interoperabil-
ity specification in order to allow build tools other than setuptools to publish pkg_resources compatible entry
point metadata, and runtime libraries other than pkg_resources to portably read published entry point metadata
(potentially with different caching and conflict resolution strategies).

Data model

Conceptually, an entry point is defined by three required properties:

• The group that an entry point belongs to indicates what sort of object it provides. For instance, the group
console_scripts is for entry points referring to functions which can be used as a command, while
[Link] is the group for classes defining pygments styles. The consumer typically defines the
expected interface. To avoid clashes, consumers defining a new group should use names starting with a PyPI
name owned by the consumer project, followed by .. Group names must be one or more groups of letters,
numbers and underscores, separated by dots (regex ^\w+(\.\w+)*$).
• The name identifies this entry point within its group. The precise meaning of this is up to the consumer. For
console scripts, the name of the entry point is the command that will be used to launch it. Within a distribution,
entry point names should be unique. If different distributions provide the same name, the consumer decides
how to handle such conflicts. The name may contain any characters except =, but it cannot start or end with
any whitespace character, or start with [. For new entry points, it is recommended to use only letters, numbers,
underscores, dashes and dots (regex [\w-.]+).

100 Chapter 5. PyPA specifications

 Python Packaging User Guide

• The object reference points to a Python object. It is either in the form [Link], or
[Link]:[Link]. Each of the parts delimited by dots and the colon is a valid Python
identifier. It is intended to be looked up like this:

import importlib
modname, qualname_separator, qualname = object_ref.partition(':')
obj = importlib.import_module(modname)
if qualname_separator:
for attr in [Link]('.'):
obj = getattr(obj, attr)

Note: Some tools call this kind of object reference by itself an ‘entry point’, for want of a better term, especially
where it points to a function to launch a program.

There is also an optional property: the extras are a set of strings identifying optional features of the distribution
providing the entry point. If these are specified, the entry point requires the dependencies of those ‘extras’. See the
metadata field Provides-Extra (multiple use).
Using extras for an entry point is no longer recommended. Consumers should support parsing them from existing
distributions, but may then ignore them. New publishing tools need not support specifying extras. The functionality of
handling extras was tied to setuptools’ model of managing ‘egg’ packages, but newer tools such as pip and virtualenv
use a different model.

File format

Entry points are defined in a file called :file:entry_points.txt in the :file:*.dist-info directory of the dis-
tribution. This is the directory described in PEP 376 for installed distributions, and in PEP 427 for wheels. The file
uses the UTF-8 character encoding.
The file contents are in INI format, as read by Python’s configparser module. However, configparser treats names
as case-insensitive by default, whereas entry point names are case sensitive. A case-sensitive config parser can be made
like this:

import configparser

class CaseSensitiveConfigParser([Link]):
optionxform = staticmethod(str)

The entry points file must always use = to delimit names from values (whereas configparser also allows using :).
The sections of the config file represent entry point groups, the names are names, and the values encode both the object
reference and the optional extras. If extras are used, they are a comma-separated list inside square brackets.
Within a value, readers must accept and ignore spaces (including multiple consecutive spaces) before or after the
colon, between the object reference and the left square bracket, between the extra names and the square brackets and
colons delimiting them, and after the right square bracket. The syntax for extras is formally specified as part of PEP
508 (as extras). For tools writing the file, it is recommended only to insert a space between the object reference and
the left square bracket.
For example:

[console_scripts]
foo = foomod:main
# One which depends on extras:
foobar = foomod:main_bar [bar,baz]
(continues on next page)

5.1. Package Distribution Metadata 101

Python Packaging User Guide

(continued from previous page)

# pytest plugins refer to a module, so there is no ':obj'

[pytest11]
nbval = [Link]

Use for scripts

Two groups of entry points have special significance in packaging: console_scripts and gui_scripts. In
both groups, the name of the entry point should be usable as a command in a system shell after the package is
installed. The object reference points to a function which will be called with no arguments when this command is run.
The function may return an integer to be used as a process exit code, and returning None is equivalent to returning 0.
For instance, the entry point mycmd = mymod:main would create a command mycmd launching a script like this:

import sys
from mymod import main
[Link](main())

The difference between console_scripts and gui_scripts only affects Windows systems.
console_scripts are wrapped in a console executable, so they are attached to a console and can use
[Link], [Link] and [Link] for input and output. gui_scripts are wrapped in a GUI
executable, so they can be started without a console, but cannot use standard streams unless application code redirects
them. Other platforms do not have the same distinction.
Install tools are expected to set up wrappers for both console_scripts and gui_scripts in the scripts direc-
tory of the install scheme. They are not responsible for putting this directory in the PATH environment variable which
defines where command-line tools are found.
As files are created from the names, and some filesystems are case-insensitive, packages should avoid using names in
these groups which differ only in case. The behaviour of install tools when names differ only in case is undefined.

5.2 Package Index Interfaces

5.2.1 Simple repository API

The current interface for querying available package versions and retrieving packages from an index server is defined
in PEP 503.

102 Chapter 5. PyPA specifications

 CHAPTER 6

Project Summaries

Summaries and links for the most relevant projects in the space of Python installation and packaging.

6.1 PyPA Projects

6.1.1 bandersnatch

Mailing list2 | Issues | Github | PyPI | Dev irc:#bandersnatch

bandersnatch is a PyPI mirroring client designed to efficiently create a complete mirror of the contents of PyPI.

6.1.2 distlib

Docs | Mailing list2 | Issues | Bitbucket | PyPI

Distlib is a library which implements low-level functions that relate to packaging and distribution of Python software.
It consists in part of the functions from the distutils2 project, which was intended to be released as packaging in the
Python 3.3 stdlib, but was removed shortly before Python 3.3 entered beta testing.

6.1.3 packaging

Docs | Dev list | Issues | Github | PyPI | User irc:#pypa | Dev irc:#pypa-dev
Core utilities for Python packaging used by pip and setuptools.
2 Multiple projects reuse the distutils-sig mailing list as their user list.

103
Python Packaging User Guide

6.1.4 pip

Docs | User list1 | Dev list | Issues | Github | PyPI | User irc:#pypa | Dev irc:#pypa-dev
A tool for installing Python packages.

6.1.5 Pipenv

Docs | Source | Issues | PyPI

Pipenv is a project that aims to bring the best of all packaging worlds to the Python world. It harnesses Pipfile, pip,
and virtualenv into one single toolchain. It features very pretty terminal colors.

6.1.6 Pipfile

Source
Pipfile and its sister [Link] are a higher-level application-centric alternative to pip’s lower-level
[Link] file.

6.1.7 Python Packaging User Guide

Docs | Mailing list | Issues | Github | User irc:#pypa | Dev irc:#pypa-dev

This guide!

6.1.8 setuptools

Docs | User list2 | Dev list | Issues | GitHub | PyPI | User irc:#pypa | Dev irc:#pypa-dev
setuptools (which includes easy_install) is a collection of enhancements to the Python distutils that allow you to
more easily build and distribute Python distributions, especially ones that have dependencies on other packages.
distribute was a fork of setuptools that was merged back into setuptools (in v0.7), thereby making setuptools the
primary choice for Python packaging.

6.1.9 twine

Mailing list2 | Issues | Github | PyPI

Twine is a utility for interacting with PyPI, that offers a secure replacement for [Link] upload.

6.1.10 virtualenv

Docs | User list | Dev list | Issues | Github | PyPI | User irc:#pypa | Dev irc:#pypa-dev
A tool for creating isolated Python environments.
1 pip was created by the same developer as virtualenv, and early on adopted the virtualenv mailing list, and it’s stuck ever since.

104 Chapter 6. Project Summaries

 Python Packaging User Guide

6.1.11 Warehouse

Docs | Mailing list2 | Issues | Github | Dev irc:#pypa-dev

The current codebase powering the Python Package Index (PyPI). It is hosted at [Link].

6.1.12 wheel

Docs | Mailing list2 | Issues | Github | PyPI | User irc:#pypa | Dev irc:#pypa-dev
Primarily, the wheel project offers the bdist_wheel setuptools extension for creating wheel distributions. Addi-
tionally, it offers its own command line utility for creating and installing wheels.

6.2 Non-PyPA Projects

6.2.1 bento

Docs | Mailing list | Issues | Github | PyPI

Bento is a packaging tool solution for Python software, targeted as an alternative to distutils, setuptools, distribute,
etc. . . . Bento’s philosophy is reproducibility, extensibility and simplicity (in that order).

6.2.2 buildout

Docs | Mailing list2 | Issues | PyPI | irc:#buildout

Buildout is a Python-based build system for creating, assembling and deploying applications from multiple parts, some
of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.

6.2.3 conda

Docs
conda is the package management tool for Anaconda Python installations. Anaconda Python is a distribution from
Anaconda, Inc specifically aimed at the scientific community, and in particular on Windows where the installation of
binary extensions is often difficult.
Conda is a completely separate tool to pip, virtualenv and wheel, but provides many of their combined features in
terms of package management, virtual environment management and deployment of binary extensions.
Conda does not install packages from PyPI and can install only from the official Anaconda repositories, or ana-
[Link] (a place for user-contributed conda packages), or a local (e.g. intranet) package server. However, note that
pip can be installed into, and work side-by-side with conda for managing distributions from PyPI.

6.2.4 devpi

Docs | Mailing List | Issues | PyPI

devpi features a powerful PyPI-compatible server and PyPI proxy cache with a complimentary command line tool to
drive packaging, testing and release activities with Python.

6.2. Non-PyPA Projects 105

Python Packaging User Guide

6.2.5 flit

Docs | Issues | PyPI

Flit is a simple way to put Python packages and modules on PyPI. Flit packages a single importable module or package
at a time, using the import name as the name on PyPI. All subpackages and data files within a package are included
automatically. Flit requires Python 3, but you can use it to distribute modules for Python 2, so long as they can be
imported on Python 3.

6.2.6 enscons

Source | Issues | PyPI

Enscons is a Python packaging tool based on SCons. It builds pip-compatible source distributions and wheels without
using distutils or setuptools, including distributions with C extensions. Enscons has a different architecture and philos-
ophy than distutils. Rather than adding build features to a Python packaging system, enscons adds Python packaging
to a general purpose build system. Enscons helps you to build sdists that can be automatically built by pip, and wheels
that are independent of enscons.

6.2.7 Hashdist

Docs | Github
Hashdist is a library for building non-root software distributions. Hashdist is trying to be “the Debian of choice for
cases where Debian technology doesn’t work”. The best way for Pythonistas to think about Hashdist may be a more
powerful hybrid of virtualenv and buildout.

6.2.8 pex

Docs | Github | PyPI

pex is both a library and tool for generating .pex (Python EXecutable) files, standalone Python environments in
the spirit of virtualenv. .pex files are just carefully constructed zip files with a #!/usr/bin/env python and
special __main__.py, and are designed to make deployment of Python applications as simple as cp.

6.2.9 pipx

Docs | Github | PyPI

pipx is a tool to safely install and run Python CLI applications globally.

6.2.10 scikit-build

Docs | Mailing list | Github | PyPI

Scikit-build is an improved build system generator for CPython C/C++/Fortran/Cython extensions that integrates with
setuptools, wheel and pip. It internally uses cmake (available on PyPI) to provide better support for additional compil-
ers, build systems, cross compilation, and locating dependencies and their associated build requirements. To speed up
and parallelize the build of large projects, the user can install ninja (also available on PyPI).

106 Chapter 6. Project Summaries

 Python Packaging User Guide

6.2.11 Spack

Docs | Github | Paper | Slides

A flexible package manager designed to support multiple versions, configurations, platforms, and compilers. Spack
is like homebrew, but packages are written in Python and parameterized to allow easy swapping of compilers, library
versions, build options, etc. Arbitrarily many versions of packages can coexist on the same system. Spack was
designed for rapidly building high performance scientific applications on clusters and supercomputers.
Spack is not in PyPI (yet), but it requires no installation and can be used immediately after cloning from github.

6.3 Standard Library Projects

6.3.1 ensurepip

Docs | Issues
A package in the Python Standard Library that provides support for bootstrapping pip into an existing Python installa-
tion or virtual environment. In most cases, end users won’t use this module, but rather it will be used during the build
of the Python distribution.

6.3.2 distutils

Docs | User list2 | Issues | User irc:#pypa | Dev irc:#pypa-dev

A package in the Python Standard Library that has support for creating and installing distributions. setuptools provides
enhancements to distutils, and is much more commonly used than just using distutils by itself.

6.3.3 venv

Docs | Issues
A package in the Python Standard Library (starting with Python 3.3) for creating Virtual Environments. For more
information, see the section on Creating Virtual Environments.

6.3. Standard Library Projects 107

Python Packaging User Guide

108 Chapter 6. Project Summaries

 CHAPTER 7

Glossary

Binary Distribution A specific kind of Built Distribution that contains compiled extensions.
Built Distribution A Distribution format containing files and metadata that only need to be moved to the correct
location on the target system, to be installed. Wheel is such a format, whereas distutil’s Source Distribution is
not, in that it requires a build step before it can be installed. This format does not imply that Python files have
to be precompiled (Wheel intentionally does not include compiled Python files).
Distribution Package A versioned archive file that contains Python packages, modules, and other resource files that
are used to distribute a Release. The archive file is what an end-user will download from the internet and install.
A distribution package is more commonly referred to with the single words “package” or “distribution”, but this
guide may use the expanded term when more clarity is needed to prevent confusion with an Import Package
(which is also commonly called a “package”) or another kind of distribution (e.g. a Linux distribution or the
Python language distribution), which are often referred to with the single term “distribution”.
Egg A Built Distribution format introduced by setuptools, which is being replaced by Wheel. For details, see The
Internal Structure of Python Eggs and Python Eggs
Extension Module A module written in the low-level language of the Python implementation: C/C++ for Python,
Java for Jython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object
(.so) file for Python extensions on Unix, a DLL (given the .pyd extension) for Python extensions on Windows,
or a Java class file for Jython extensions.
Known Good Set (KGS) A set of distributions at specified versions which are compatible with each other. Typically
a test suite will be run which passes all tests before a specific set of packages is declared a known good set. This
term is commonly used by frameworks and toolkits which are comprised of multiple individual distributions.
Import Package A Python module which can contain other modules or recursively, other packages.
An import package is more commonly referred to with the single word “package”, but this guide will use the
expanded term when more clarity is needed to prevent confusion with a Distribution Package which is also
commonly called a “package”.
Module The basic unit of code reusability in Python, existing in one of two types: Pure Module, or Extension Module.
Package Index A repository of distributions with a web interface to automate package discovery and consumption.

109
Python Packaging User Guide

Per Project Index A private or other non-canonical Package Index indicated by a specific Project as the index pre-
ferred or required to resolve dependencies of that project.
Project A library, framework, script, plugin, application, or collection of data or other resources, or some combination
thereof that is intended to be packaged into a Distribution.
Since most projects create Distributions using distutils or setuptools, another practical way to define projects
currently is something that contains a [Link] at the root of the project src directory, where “[Link]” is the
project specification filename used by distutils and setuptools.
Python projects must have unique names, which are registered on PyPI. Each project will then contain one or
more Releases, and each release may comprise one or more distributions.
Note that there is a strong convention to name a project after the name of the package that is imported to run
that project. However, this doesn’t have to hold true. It’s possible to install a distribution from the project ‘foo’
and have it provide a package importable only as ‘bar’.
Pure Module A module written in Python and contained in a single .py file (and possibly associated .pyc and/or .pyo
files).
Python Packaging Authority (PyPA) PyPA is a working group that maintains many of the relevant projects in
Python packaging. They maintain a site at [Link] host projects on github and bitbucket, and
discuss issues on the pypa-dev mailing list.
Python Package Index (PyPI) PyPI is the default Package Index for the Python community. It is open to all Python
developers to consume and distribute their distributions.
[Link] [Link] is the domain name for the Python Package Index (PyPI). It replaced the legacy index domain
name, [Link], in 2017. It is powered by Warehouse.
Release A snapshot of a Project at a particular point in time, denoted by a version identifier.
Making a release may entail the publishing of multiple Distributions. For example, if version 1.0 of a project
was released, it could be available in both a source distribution format and a Windows installer distribution
format.
Requirement A specification for a package to be installed. pip, the PYPA recommended installer, allows various
forms of specification that can all be considered a “requirement”. For more information, see the pip install
reference.
Requirement Specifier A format used by pip to install packages from a Package Index. For an EBNF diagram of
the format, see the pkg_resources.Requirement entry in the setuptools docs. For example, “foo>=1.3” is a
requirement specifier, where “foo” is the project name, and the “>=1.3” portion is the Version Specifier
Requirements File A file containing a list of Requirements that can be installed using pip. For more information, see
the pip docs on Requirements Files.
[Link] The project specification file for distutils and setuptools.
Source Archive An archive containing the raw source code for a Release, prior to creation of an Source Distribution
or Built Distribution.
Source Distribution (or “sdist”) A distribution format (usually generated using python [Link] sdist)
that provides metadata and the essential source files needed for installing by a tool like pip, or for generating a
Built Distribution.
System Package A package provided in a format native to the operating system, e.g. an rpm or dpkg file.
Version Specifier The version component of a Requirement Specifier. For example, the “>=1.3” portion of
“foo>=1.3”. PEP 440 contains a full specification of the specifiers that Python packaging currently supports.
Support for PEP440 was implemented in setuptools v8.0 and pip v6.0.

110 Chapter 7. Glossary

 Python Packaging User Guide

Virtual Environment An isolated Python environment that allows packages to be installed for use by a particular
application, rather than being installed system wide. For more information, see the section on Creating Virtual
Environments.
Wheel A Built Distribution format introduced by PEP 427, which is intended to replace the Egg format. Wheel is
currently supported by pip.
Working Set A collection of distributions available for importing. These are the distributions that are on the [Link]
variable. At most, one Distribution for a project is possible in a working set.

111
Python Packaging User Guide

112 Chapter 7. Glossary

 CHAPTER 8

How to Get Support

For support related to a specific project, see the links on the Projects page.
For something more general, or when you’re just not sure, use the distutils-sig list.

113
Python Packaging User Guide

114 Chapter 8. How to Get Support

 CHAPTER 9

Contribute to this guide

The Python Packaging User Guide welcomes contributors! There are lots of ways to help out, including:
• Reading the guide and giving feedback
• Reviewing new contributions
• Revising existing content
• Writing new content
Most of the work on the Python Packaging User Guide takes place on the project’s GitHub repository. To get started,
check out the list of open issues and pull requests. If you’re planning to write or edit the guide, please read the style
guide.
By contributing to the Python Packaging User Guide, you’re expected to follow the Python Packaging Authority’s
Contributor Code of Conduct. Harassment, personal attacks, and other unprofessional conduct is not acceptable.

9.1 Documentation types

This project consists of four distinct documentation types with specific purposes. When proposing new additions to
the project please pick the appropriate documentation type.

9.1.1 Tutorials

Tutorials are focused on teaching the reader new concepts by accomplishing a goal. They are opinionated step-by-step
guides. They do not include extraneous warnings or information. example tutorial-style document.

9.1.2 Guides

Guides are focused on accomplishing a specific task and can assume some level of pre-requisite knowledge. These
are similar to tutorials, but have a narrow and clear focus and can provide lots of caveats and additional information as
needed. They may also discuss multiple approaches to accomplishing the task. example guide-style document.

115
Python Packaging User Guide

9.1.3 Discussions

Discussions are focused on understanding and information. These explore a specific topic without a specific goal in
mind. example discussion-style document.

9.1.4 Specifications

Specifications are reference documention focused on comprehensively documenting an agreed-upon iterface for inter-
operability between packaging tools. example specification-style document.

9.2 Building the guide locally

Though not required to contribute, it may be useful to build this guide locally in order to test your changes. In order
to build this guide locally, you’ll need:
1. Nox. You can install or upgrade nox using pip:

pip install --user nox

2. Python 3.6. Our build scripts are designed to work with Python 3.6 only. See the Hitchhiker’s Guide to Python
installation instructions to install Python 3.6 on your operating system.
To build the guide, run the following bash command in the source folder:

nox -s build

After the process has completed you can find the HTML output in the ./build/html directory. You can open the
[Link] file to view the guide in web browser, but it’s recommended to serve the guide using an HTTP server.
You can build the guide and serve it via an HTTP server using the following command:

nox -s preview

The guide will be browsable via [Link]

9.3 Where the guide is deployed

The guide is deployed via ReadTheDocs and the configuration lives at [Link]
python-packaging-user-guide/. It’s served from a custom domain and fronted by [Link].

9.4 Style guide

This style guide has recommendations for how you should write the Python Packaging User Guide. Before you start
writing, please review it. By following the style guide, your contributions will help add to a cohesive whole and make
it easier for your contributions to be accepted into the project.

116 Chapter 9. Contribute to this guide

 Python Packaging User Guide

9.4.1 Purpose

The purpose of the Python Packaging User Guide is

to be the authoritative resource on how to package, publish, and install Python projects using current tools.

9.4.2 Scope

The guide is meant to answer questions and solve problems with accurate and focused recommendations.
The guide isn’t meant to be comprehensive and it’s not meant to replace individual projects’ documentation. For
example, pip has dozens of commands, options, and settings. The pip documentation describes each of them in detail,
while this guide describes only the parts of pip that are needed to complete the specific tasks described in this guide.

9.4.3 Audience

The audience of this guide is anyone who uses Python with packages.
Don’t forget that the Python community is big and welcoming. Readers may not share your age, gender, education,
culture, and more, but they deserve to learn about packaging just as much as you do.
In particular, keep in mind that not all people who use Python see themselves as programmers. The audience of this
guide includes astronomers or painters or students as well as professional software developers.

9.4.4 Voice and tone

When writing this guide, strive to write with a voice that’s approachable and humble, even if you have all the answers.
Imagine you’re working on a Python project with someone you know to be smart and skilled. You like working with
them and they like working with you. That person has asked you a question and you know the answer. How do you
respond? That is how you should write this guide.
Here’s a quick check: try reading aloud to get a sense for your writing’s voice and tone. Does it sound like something
you would say or does it sound like you’re acting out a part or giving a speech? Feel free to use contractions and don’t
worry about sticking to fussy grammar rules. You are hereby granted permission to end a sentence in a preposition, if
that’s what you want to end it with.
When writing the guide, adjust your tone for the seriousness and difficulty of the topic. If you’re writing an introductory
tutorial, it’s OK to make a joke, but if you’re covering a sensitive security recommendation, you might want to avoid
jokes altogether.

9.4.5 Conventions and mechanics

Write to the reader When giving recommendations or steps to take, address the reader as you or use the imperative
mood.

Wrong: To install it, the user runs. . .

Right: You can install it by running. . .
Right: To install it, run. . .

9.4. Style guide 117

Python Packaging User Guide

State assumptions Avoid making unstated assumptions. Reading on the web means that any page of the guide may
be the first page of the guide that the reader ever sees. If you’re going to make assumptions, then say what
assumptions that you’re going to make.
Cross-reference generously The first time you mention a tool or practice, link to the part of the guide that covers it,
or link to a relevant document elsewhere. Save the reader a search.
Respect naming practices When naming tools, sites, people, and other proper nouns, use their preferred capitaliza-
tion.

Wrong: Pip uses. . .

Right: pip uses. . .

Wrong: . . . hosted on github.

Right: . . . hosted on GitHub.

Use a gender-neutral style Often, you’ll address the reader directly with you, your and yours. Otherwise, use gender-
neutral pronouns they, their, and theirs or avoid pronouns entirely.

Wrong: A maintainer uploads the file. Then he. . .

Right: A maintainer uploads the file. Then they. . .
Right: A maintainer uploads the file. Then the maintainer. . .

Headings Write headings that use words the reader is searching for. A good way to do this is to have your heading
complete an implied question. For example, a reader might want to know How do I install MyLibrary? so a
good heading might be Install MyLibrary.
In section headings, use sentence case. In other words, write headings as you would write a typical sentence.

Wrong: Things You Should Know About Python

Right: Things you should know about Python

Numbers In body text, write numbers one through nine as words. For other numbers or numbers in tables, use
numerals.

118 Chapter 9. Contribute to this guide

 CHAPTER 10

News

10.1 September 2018

• Added a section on checking RST markup. (#554)

• Updated user installs page. (#558)
• Updated Google BigQuery urls. (#556)
• Replaced tar command with working command. (#552)
• Changed to double quotes in the pip install SomeProject==1.4. (#550)

10.2 August 2018

• Removed the recommendation to store passwords in cleartext. (#546)

• Moved the Overview to a task based lead in along with the others. (#540)
• Updated Python version supported by virtualenv. (#538)
• Added outline/rough draft of new Overview page. (#519)

10.3 July 2018

• Improved binary extension docs. (#531)

• Added scikit-build to key projects. (#530)

119
Python Packaging User Guide

10.4 June 2018

• Fixed categories of interop PEP for [Link]. (#527)

• Updated Markdown descriptions explanation. (#522)

10.5 May 2018

• Noted issues with Provides-Dist and Obsoletes-Dist. (#513)

• Removed outdated warning about Python version mixing with Pipenv. (#501)
• Simplified packaging tutorial. (#498)
• Updated Windows users instructions for clarity. (#493)
• Updated the license section description for completeness. (#492)
• Added specification-style document to contributing section. (#489)
• Added documentation types to contributing guide. (#485)

10.6 April 2018

• Added README guide. (#461)

• Updated instructions and status for PyPI launch. (#475)
• Added instructions for Warehouse. (#471)
• Removed GPG references from publishing tutorial. (#466)
• Added ‘What’s in which Python 3.4–3.6?’. (#468)
• Added a guide for phasing out Python versions. (#459)
• Made default Description-Content-Type variant GFM. (#462)

10.7 March 2018

• Updated “installing scientific packages”. (#455)

• Added long_description_content_type to follow PEP 556. (#457)
• Clarified a long description classifier on [Link]. (#456)
• Updated Core Metadata spec to follw PEP 556. (#412)

10.8 February 2018

• Added python3-venv and python3-pip to Debian installation instructions. (#445)

• Updated PyPI migration info. (#439)
• Added a warning about managing multiple versions with pipenv. (#430)

120 Chapter 10. News

 Python Packaging User Guide

• Added example of multiple emails to Core Metadata. (#429)

• Added explanation of “legacy” in [Link]/legacy. (#426)

10.9 January 2018

• Added a link to PyPI’s list of classifiers. (#425)

• Updated [Link] explanation. (#419)

10.10 December 2017

• Replaced ~ with $HOME in guides and tutorials. (#418)

• Noted which fields can be used with environment markers. (#416)
• Updated Requires-Python section. (#414)
• Added news page. (#404)

10.11 November 2017

• Introduced a new dependency management tutorial based on Pipenv. (#402)

• Updated the Single Sourcing Package Version tutorial to reflect pip’s current strategy. (#400)
• Added documentation about the py_modules argument to setup. (#398)
• Simplified the wording for the [Link] section. (#395)

10.12 October 2017

• Added a specification for the entry_points.txt file. (#398)

• Created a new guide for managing packages using pip and virtualenv. (#385)
• Split the specifications page into multiple pages. (#386)

10.13 September 2017

• Encouraged using readme_renderer to validate [Link]. (#379)

• Recommended using the –user-base option. (#374)

10.14 August 2017

• Added a new, experimental tutorial on installing packages using Pipenv. (#369)

• Added a new guide on how to use TestPyPI. (#366)
• Added [Link] as a term. (#365)

10.9. January 2018 121

Python Packaging User Guide

10.15 July 2017

• Added flit to the key projects list. (#358)

• Added enscons to the list of key projects. (#357)
• Updated this guide’s readme with instructions on how to build the guide locally. (#356)
• Made the new TestPyPI URL more visible, adding note to homepage about [Link]. (#354)
• Added a note about the removal of the explicit registration API. (#347)

10.16 June 2017

• Added a document on migrating uploads to [Link]. (#339)

• Added documentation for python_requires. (#338)
• Added a note about PyPI migration in the Tool Recommendations tutorial. (#335)
• Added a note that [Link] does not affect wheels. (#332)
• Added a license section to the distributing guide. (#331)
• Expanded the section on the name argument. (#329)
• Adjusted the landing page. (#327, #326, #324)
• Updated to Sphinx 1.6.2. (#323)
• Switched to the PyPA theme. (#305)
• Re-organized the documentation into the new structure. (#318)

10.17 May 2017

• Added documentation for the Description-Content-Type field. (#258)

• Added contributor and style guide. (#307)
• Documented pip and easy_install’s differences for per-project indexes. (#233)

10.18 April 2017

• Added travis configuration for testing pull requests. (#300)

• Mentioned the requirement of the wheel package for creating wheels (#299)
• Removed the twine register reference in the Distributing Packages tutorial. (#271)
• Added a topic on plugin discovery. (#294, #296)
• Added a topic on namespace packages. (#290)
• Added documentation explaining prominently how to install pip in /usr/local. (#230)
• Updated development mode documentation to mention that order of local packages matters. (#208)
• Convert readthedocs link for their .org -> .io migration for hosted projects (#239)

122 Chapter 10. News

 Python Packaging User Guide

• Swaped order of [Link] arguments for the upload command, as order is significant. (#260)
• Explained how to install from unsupported sources using a helper application. (#289)

10.19 March 2017

• Covered manylinux1 in Platform Wheels. (#283)

10.20 February 2017

• Added PEP 518. (#281)

Welcome to the Python Packaging User Guide, a collection of tutorials and references to help you distribute and install
Python packages with modern tools.
This guide is maintained on GitHub by the Python Packaging Authority. We happily accept any contributions and
feedback.

10.19. March 2017 123

Python Packaging User Guide

124 Chapter 10. News

 CHAPTER 11

Get started

Essential tools and concepts for working within the Python development ecosystem are covered in our Tutorials
section:
• to learn how to install packages, see the tutorial on installing packages.
• to learn how to manage dependencies in a version controlled project, see the tutorial on managing application
dependencies.
• to learn how to package and distribute your projects, see the tutorial on packaging and distributing
• to get an overview of packaging options for Python libraries and applications, see the Overview of Python
Packaging.

125
Python Packaging User Guide

126 Chapter 11. Get started

 CHAPTER 12

Learn more

Beyond our Tutorials, this guide has several other resources:

• the Guides section for walk throughs, such as Installing pip/setuptools/wheel with Linux Package Managers or
Packaging binary extensions
• the Discussions section for in-depth references on topics such as Deploying Python applications or pip vs
easy_install
• the PyPA specifications section for packaging interoperability specifications
Additionally, there is a list of other projects maintained by members of the Python Packaging Authority.

127
Python Packaging User Guide

128 Chapter 12. Learn more

 Index

B PEP 427#is-it-possible-to-import-python-code-dir
Binary Distribution, 109 87
Built Distribution, 109 PEP 440, 13, 19, 44, 46, 90, 99, 110
PEP 440#compatible-release, 13, 47
D PEP 440#local-version-identifiers,
Distribution Package, 109 47
PEP 440#normalization, 46
E PEP 440#public-version-identifiers,
Egg, 109 46
environment variable PEP 440#version-specifiers, 13, 110
PATH, 13, 14 PEP 453, 25
Extension Module, 109 PEP 453#rationale, 25
PEP 503, 15, 102
I PEP 508, 41, 91, 96, 99, 101
PEP 513, 50, 100
Import Package, 109
PEP 517, 99
K PEP 518, 99, 123
PEP 566, 89
Known Good Set (KGS), 109
PEP 571, 100
Python Package Index (PyPI), 110
M Python Packaging Authority (PyPA), 110
Module, 109
R
P Release, 110
Package Index, 109 Requirement, 110
PATH, 13, 14 Requirement Specifier, 110
Per Project Index, 110 Requirements File, 110
Project, 110
Pure Module, 110 S
[Link], 110 [Link], 110
Python Enhancement Proposals Source Archive, 110
PEP 438, 85 Source Distribution (or "sdist"), 110
Python Enhancement Proposals System Package, 110
PEP 301, 95
PEP 345, 99 V
PEP 376, 87, 100, 101 Version Specifier, 110
PEP 397, 46 Virtual Environment, 111
PEP 420, 67
PEP 425, 50, 87, 99, 100 W
PEP 427, 87, 99, 101, 111 Wheel, 111

129
Python Packaging User Guide

Working Set, 111

130 Index