Introducing Python packages

In this lesson, we'll investigate what we mean when we refer to Python packages. We'll consider how we use built-in Python packages, as well as mention some standard Python libraries and their
applications.

Learning objectives
In this train, we will learn how to:

Define a package and its hierarchical organisation.

What are packages in Python?

For obvious reasons, we can't really store all of our files on our computer in the same location or folder. It would be nearly impossible to find the relevant file we're looking for! Therefore, we make use
of well-organised directory structures for easier accessibility.

A specific directory is designated to files that share similarities, for example, we may keep all the photos in the "Pictures" directory. In the same way,
Python packages can be seen as structures
similar to directories with modules playing a role similar to files.

As our program grows larger in size with an increased number of modules, we can cluster similar modules in one package and other clusters of similar modules in different packages. This, in turn, will
allow for the efficient management of our project (program), making it conceptually clear. Similarly, as a directory can contain subdirectories and files, a Python package can also contain sub-packages
and modules.

This organisation of modules and code allows us to define aPython package (also referred to as a library) as a collection of hierarchically structured directories of Python code, consisting of
sub-packages and modules.

In order for a directory to be considered as a package by Python, it must contain a file named__init__.py. This file can be left empty but the initialisation code for that package is generally placed in this
file.

The figure below presents a possible organisation of packages and modules present if we were developing a game:

Figure 1: In this figure, we use a game to represent the package structure used in Python, allowing us to logically structure our code as it
grows in complexity.
Standard Python packages
Python distributions are shipped with a standard list of libraries/packages. Some of these include:

It's not crucial to know what all these packages do right now, but it is useful to know that they exist, so feel free to read up on them in more detail later.

Text processing services

string – Common string operations

re – Regular expression operations
unicodedata – Unicode Database

Data types

datetime – Basic date and time types

calendar – General calendar-related functions
array – Efficient arrays of numeric values
copy – Shallow and deep copy operations
pprint – Data pretty printer

Numeric and mathematical modules

numbers – Numeric abstract base classes

math – Mathematical functions
cmath – Mathematical functions for complex numbers
decimal – Decimal fixed point and floating point arithmetic
fractions – Rational numbers
 random – Generate pseudo-random numbers
statistics – Mathematical statistics functions

File and directory access

pathlib – Object-oriented filesystem paths

fileinput – Iterate over lines from multiple input streams
stat – Interpreting stat() results
filecmp – File and directory comparisons
tempfile – Generate temporary files and directories
shutil – High-level file operations

Data persistence

pickle – Python object serialisation

copyreg – Register pickle support functions
shelve – Python object persistence
marshal – Internal Python object serialisation
dbm – Interfaces to Unix “databasesâ€​
sqlite3 – DB-API 2.0 interface for SQLite databases

Data compression and archiving

zlib – Compression compatible with gzip

gzip – Support for gzip files
bz2 – Support for bzip2 compression
lzma – Compression using the LZMA algorithm
zipfile – Work with ZIP archives
tarfile – Read and write tar archive files

File formats

csv – CSV file reading and writing

configparser – Configuration file parser
netrc – netrc file processing
xdrlib – Encode and decode XDR data
plistlib – Generate and parse Mac OS X .plist files

Cryptographic services

hashlib – Secure hashes and message digests

hmac – Keyed hashing for message authentication
secrets – Generate secure random numbers for managing secrets

Generic operating system services

os – Miscellaneous operating system interfaces

io – Core tools for working with streams
time – Time access and conversions
errno – Standard errno system symbols
ctypes – A foreign function library for Python

Concurrent execution

threading – Thread-based parallelism

multiprocessing – Process-based parallelism
subprocess – Subprocess management

Networking and interprocess communication

asyncio – Asynchronous I/O

socket – Low-level networking interface
ssl – TLS/SSL wrapper for socket objects
signal – Set handlers for asynchronous events
mmap – Memory-mapped file support

Internet data handling

email – An email and MIME handling package

json – JSON encoder and decoder
mailcap – Mailcap file handling
mailbox – Manipulate mailboxes in various formats
Graphical User Interfaces with Tk
tkinter – Python interface to Tcl/Tk

Python packages can also be installed from local or online repositories such as thePackage Index (PyPI), a repository of software for the Python programming language.

PyPI helps you find and install software developed and shared by the Python community. For specific applications such as scientific computing, packages can be installed using package managers
such as Anaconda.

Conclusion
In this train, we looked at Python packages; their creation and hierarchical organisation. There are endless uses of modules and packages, and each can be tailored to an area of expertise and shared
with other programmers.

We can apply the skills we have gained from this train to create our own custom-built modules and packages. We can even create a package that may assist with data analysis!

Appendix
Below are additional useful resources to help you further under Python modules and packages:

Official Python Tutorial for Modules