Module 5: Advanced Topics in Python

REGULAR EXPRESSIONS
Regular Expression (RegEx) is a powerful tool used to search, match, validate, extract, or
modify text based on specific patterns. In Python, the built-in re module provides support for
using RegEx. Regular expressions are widely used in various fields involving text manipulation
and analysis. Below are some common use cases:
• Data Mining: Quickly extract emails, phone numbers, URLs, etc. from large text
blocks.
• Validation: Validate user inputs like email addresses, passwords, dates, etc.
• Text Processing: Replace or reformat strings to match required formats (e.g., date
reformatting).

Importing re Package: In Python, regular expressions are supported by the re module. The
basic syntax for working with regular expressions in Python is as: import re

Table 5.1: Metacharacters in Regular Expressions

Meta
Description
character
[] Represent a character class
^ Matches the beginning
$ Matches the end
. Matches any character except newline
? Matches zero or one occurrence.
| Means OR (Matches with any of the characters separated by it.
* Any number of occurrences (including 0 occurrences)
+ One or more occurrences
{} Indicate the number of occurrences of a preceding RE to match.
() Enclose a group of REs

Searching for a pattern using [Link]() Method: [Link]() method either returns None
(if the pattern doesn’t match), or a [Link] that contains information about the matching
part of the string. This method stops after the first match, so this is best suited for testing a
regular expression more than extracting data.
 Table 5.2: RegEx Functions
Function Description
findall Returns a list containing all matches
search Returns a Match object if there is a match anywhere in the string
split Returns a list where the string has been split at each match
sub Replaces one or many matches with a string

Regular expressions are a powerful tool for working with strings and text data in Python.
Whether you're matching patterns, replacing text, or extracting information, regular
expressions make it easy to perform complex string operations with just a few lines of code.
With a little bit of practice, you'll be able to use regular expressions to solve all sorts of string-
related problems in Python.

WORKING WITH DATABASES

Python, being a high-level language, provides support for various databases. We can connect
and run queries for a particular database using Python, and without writing raw queries in the
terminal or shell of that particular database, we just need to have that database installed in our
system.
A database is an organized collection of data stored electronically. It is managed by a Database
Management System (DBMS) that allows users to access, update, and manage data efficiently.
Databases are primarily categorized into two types:
1. Relational Databases (RDBMS): Store data in structured tables with rows and
columns. They support relationships between data entities. Examples:
a. MySQL
b. PostgreSQL
c. SQLite
2. Non-relational Databases (NoSQL): Store data in various formats like documents,
key-value pairs, or graphs, offering more flexibility. For Examples:
a. MongoDB
b. Cassandra

Python MySQL: Python MySQL Connector is a Python driver that helps to integrate Python
and MySQL. This Python MySQL library allows the conversion between Python and MySQL
data types. MySQL Connector API is implemented using pure Python and does not require any
third-party libraries.
Python SQLite: The Python SQLite3 module is used to integrate the SQLite database with
Python. It is a standardized Python DBI API 2.0 and provides a straightforward and simple-to-
use interface for interacting with SQLite databases. There is no need to install this module
separately, as it comes along with Python after the 2.5x version.

MULTITHREADING AND MULTIPROCESSING

Multiprocessing
It refers to the ability of a system to support more than one processor simultaneously.
Applications in a multiprocessing system are broken down into smaller routines that run
independently. The operating system allocates these threads to the processors, thereby
improving the system's performance.

Role of Multiprocessing in Python: Consider a computer system with a single processor. If it

is assigned several processes at the same time, it will have to interrupt each task and switch
briefly to another, to keep all of the processes going. This situation is just like a chef working
in a kitchen alone. He has to do several tasks like baking, stirring, and kneading dough, etc. So
the gist is that: The more tasks you must do at once, the more difficult it gets to keep track of
them all, and keeping the timing right becomes more of a challenge. This is where the concept
of multiprocessing arises.
A multiprocessing system can have:
• multiprocessor, i.e., a computer with more than one central processor.
• multi-core processor, i.e., a single computing component with two or more independent
actual processing units (called "cores").
In Python, a process is an instance of a program in execution. Each process has its own memory
space, system resources, and execution context. By using multiprocessing, you can distribute
tasks across multiple processes, leveraging the power of multiple CPU cores in your machine.

Multithreading
It allows you to run multiple threads concurrently within a single process, which is also known
as thread-based parallelism. This means a program can perform multiple tasks at the same time,
enhancing its efficiency and responsiveness. Multithreading in Python is especially useful for
multiple I/O-bound operations, rather than for tasks that require heavy computation.
Generally, a computer program sequentially executes the instructions from start to end.
Whereas, Multithreading divides the main task into more than one sub-task and executes them
in an overlapping manner.

Process: A process is what we call a program that has been loaded into memory along with all
the resources it needs to operate. It has its own memory space.
Thread: A thread is the unit of execution within a process. A process can have multiple threads
running as a part of it, where each thread uses the process’s memory space and shares it with
other threads.
An operating system is capable of handling multiple processes concurrently. It allocates a
separate memory space to each process so that one process cannot access or write anything in
another's space. On the other hand, a thread can be considered a lightweight sub-process in a
single program that shares the memory space allocated to it, facilitating easier communication
and data sharing. As they are lightweight and do not require much memory overhead, they are
cheaper than processes. A process always starts with a single thread (main thread). As and when
required, a new thread can be started, and sub task is delegated to it. Now the two threads are
working in an overlapping manner. When the task assigned to the secondary thread is over, it
merges with the main thread. A thread has a beginning, an execution sequence, and a
conclusion. It has an instruction pointer that keeps track of where it is currently running within
its context.

GUI PROGRAMMING
Python provides various options for developing Graphical User Interfaces (GUIs).
• Tkinter: It is the Python interface to the Tk GUI toolkit shipped with Python.
• wxPython: This is an open-source Python interface for the wxWidgets GUI toolkit.
• PyQt: This is also a Python interface for a popular cross-platform Qt GUI library.
• PyGTK: It is a set of wrappers written in Python and C for the GTK+ GUI library.
• PySimpleGUI: It is an open-source, cross-platform GUI library for Python. It aims to
provide a uniform API for creating desktop GUIs based on Python's Tkinter, PySide,
and WxPython toolkits.

Tkinter Programming
Tkinter is the standard GUI library for Python. Python, when combined with Tkinter, provides
a fast and easy way to create GUI applications. Tkinter provides a powerful object-oriented
interface to the Tk GUI toolkit.
The tkinter package includes the following modules.
• Tkinter − Main Tkinter module.
• [Link] − Dialog to let the user choose a color.
• [Link] − Common dialogs to allow the user to specify a file to open or save.
• [Link] − Utilities to help work with fonts.
• [Link] − Access to standard Tk dialog boxes.
• [Link] − Text widget with a vertical scroll bar built in.
• [Link] − Basic dialogs and convenience functions.

Tkinter Widgets: Tkinter provides various controls, such as buttons, labels, and text boxes,
used in a GUI application. These controls are commonly called widgets.
• Button: The Button widget is used to display the buttons in your application.
• Canvas: It is used to draw shapes, such as lines, ovals, polygons, and rectangles, in your
application.
• Checkbutton: It is used to display several options as checkboxes. The user can select
multiple options at a time.
• Entry: It is used to display a single-line text field for accepting values from a user.
• Frame: It is used as a container widget to organize other widgets.
• Label: It is used to provide a single-line caption for other widgets. It can also contain
images.
• Listbox: It is used to provide a list of options to a user.
• Menubutton: It is used to display menus in your application.
• Menu: It is used to provide various commands to a user. These commands are contained
inside Menubutton.
• Message: It is used to display multiline text fields for accepting values from a user.
• Radiobutton: It is used to display several options as radio buttons. The user can select
only one option at a time.
• Scale: It is used to provide a slider widget.
• Scrollbar: It is used to add scrolling capability to various widgets, such as list boxes.
• Text: It is used to display text in multiple lines.
• Toplevel: It is used to provide a separate window container.
• LabelFrame: It is a simple container widget. Its primary purpose is to act as a spacer or
container for complex window layouts.
 • tkMessageBox: This module is used to display message boxes in your applications.

WEB SCRAPING
Web scraping is the process of extracting data from web pages using scripts or programs.
Python provides various libraries for web scraping, such as BeautifulSoup, Scrapy, and
Requests.
• Web Scraping Tools: Python provides several libraries for web scraping, each with its
own advantages and disadvantages. The most popular libraries include BeautifulSoup,
Scrapy, and Requests.
• HTML and CSS: HTML (Hypertext Markup Language) and CSS (Cascading Style
Sheets) are the building blocks of web pages. Understanding the basics of HTML and
CSS is crucial for web scraping.
• XPath and CSS Selectors: XPath and CSS selectors are used to navigate through the
HTML structure of web pages and locate specific elements. These selectors can be used
in conjunction with web scraping libraries to extract data.
• HTTP Requests and Responses: HTTP requests and responses are the communication
protocols used by web browsers and servers. Understanding these protocols is essential
for web scraping.
• Data Extraction: Once the relevant data has been identified, it can be extracted from
the web page and processed for further analysis.

INTRODUCTION TO DATA VISUALIZATION

In today's world, a lot of data is being generated daily. And sometimes, to analyze this data for
certain trends, patterns may become difficult if the data is in its raw format. To overcome this,
data visualization comes into play. Data visualization provides a good, organized pictorial
representation of the data, which makes it easier to understand, observe, and analyze.

Matplotlib
Matplotlib is a Python library used for creating static, animated, and interactive data
visualizations. It is built on top of NumPy, and it can easily handle large datasets for creating
various types of plots, such as line charts, bar charts, scatter plots, etc.

Visualizing Data with Pyplot using Matplotlib

Pyplot is a module in Matplotlib that provides a simple interface for creating plots. It allows
users to generate charts like line graphs, bar charts, and histograms with minimal code.
• Line Chart: It is one of the basic plots and can be created using the plot() function. It
is used to represent a relationship between two data X and Y on different axis.
• Bar Chart: It displays categorical data using rectangular bars whose lengths are
proportional to the values they represent. It can be plotted vertically or horizontally to
compare different categories.
• Histogram: It shows the distribution of data by grouping values into bins. The hist()
function is used to create it, with the X-axis showing bins and the Y-axis showing
frequencies.
• Scatter Plot: These are used to observe relationships between variables. The scatter()
method in the matplotlib library is used to draw a scatter plot.
• Pie Chart: It is a circular chart used to show data as proportions or percentages. It is
created using the pie(), where each slice (wedge) represents a part of the whole.
• Box Plot: It is a simple graph that shows how data is spread out. It displays the
minimum, maximum, median, and quartiles, and also helps to spot outliers easily.
• Heatmap: It is a graphical representation of data where values are shown as colors. It
helps visualize patterns, correlations, or intensity in a matrix-like format. It is created
using the imshow() method in Matplotlib.