PYTHON

FROM BEGINNER TO ADVANCED

Complete Edition — All Topics, All Levels

Part I — Core Language

Variables · OOP · Generators · Decorators · Standard Library
Part II — Advanced Python
Type Hints · Async/Await · Concurrency · Metaclasses ·
Descriptors
Part III — Professional Python
Testing · Packaging · Design Patterns · Memory &
Performance

24 Chapters · 300+ Code Examples · 180+ Practice Exercises

 PART I
Core Language
Variables through the Standard Library
Chapter 1
Variables & Data Types
The foundation of every Python program

Python is dynamically typed — you assign a value, Python infers the type. Variables are just
names pointing to objects in memory.

1.1 Variables & Assignment

name = "Alice" # str
age = 30 # int
height = 5.9 # float
active = True # bool
nothing = None # NoneType

# Multiple assignment
x = y = z = 0
a, b, c = 1, 2, 3 # unpacking
a, b = b, a # swap in one line
first, *rest = [1,2,3,4,5] # starred assignment
# first=1, rest=[2,3,4,5]

1.2 int — Integer

Arbitrary precision whole numbers. Python 3 integers never overflow.
x = 42
big = 10_000_000 # underscores for readability
binary = 0b1010 # 10
octal = 0o17 # 15
hexval = 0xFF # 255
print(type(x)) # <class 'int'>
print(isinstance(x,int)) # True

1.3 float, complex

pi = 3.14159
sci = 1.5e-3 # 0.0015
inf = float('inf')
nan = float('nan')
 # Precision warning
print(0.1 + 0.2) # 0.30000000000000004
from decimal import Decimal
print(Decimal("0.1") + Decimal("0.2")) # 0.2 (exact)

# Complex numbers
z = 3 + 4j
print([Link], [Link]) # 3.0 4.0
print(abs(z)) # 5.0 (magnitude)

1.4 str — String

s = 'Hello, World!'
multi = """Line one
Line two"""

# f-strings (3.6+)
name = "Bob"
msg = f"Hello {name!r}, age {2024-1994}"

# String methods (sample)

print("hello".upper()) # HELLO
print(" hi ".strip()) # hi
print("a,b,c".split(",")) # ['a','b','c']
print("abc".replace("b","X")) # aXc
print("python".center(11,"-")) # --python---
print(",".join(["a","b","c"])) # a,b,c

# Slicing
s = "abcdefgh"
print(s[2:5]) # cde
print(s[::-1]) # hgfedcba
print(s[::2]) # aceg

1.5 list, tuple, set, dict

# list — ordered, mutable
fruits = ["apple","banana","cherry"]
[Link]("date")
[Link](1,"avocado")
 [Link]()
print(fruits[0], fruits[-1])

# tuple — ordered, immutable

coords = (10, 20, 30)
x, y, z = coords # unpack

# set — unordered, unique

primes = {2, 3, 5, 7}
[Link](11)
print(3 in primes) # True
a={1,2,3}; b={2,3,4}
print(a & b, a | b, a - b) # {2,3} {1,2,3,4} {1}

# dict — key-value
person = {"name":"Alice","age":30}
person["email"] = "a@[Link]"
print([Link]("phone","N/A")) # N/A
for k,v in [Link](): print(f"{k}: {v}")

1.6 Type Conversion & Inspection

# Conversion
print(int("42")) # 42
print(float("3.14")) # 3.14
print(str(100)) # "100"
print(list("abc")) # ['a','b','c']
print(bool(0), bool([])) # False False

# Inspection
x = [1, 2, 3]
print(type(x)) # <class 'list'>
print(isinstance(x, list))# True
print(id(x)) # memory address

💡 Mutable vs Immutable
str, int, float, bool, tuple, frozenset are immutable — operations return new objects. list, dict,
set are mutable — modified in place.
✏ Practice Exercises
1. Create variables for your name, age, height, and a list of 3 hobbies. Print each with
type().
2. Demonstrate starred assignment: unpack [1,2,3,4,5] into first, middle (list), last.
3. Show 5 string methods on " The Quick Brown Fox " — strip, lower, split, count,
replace.
4. Create two sets (favourite movies, recommended movies). Find overlap, movies only
you like, and the union.
5. Show why 0.1 + 0.2 != 0.3 and fix it using the decimal module.
6. Build a dict of a person, iterate over it, update two fields, and delete one key.
7. Write a program using all numeric types: int, float, complex — perform arithmetic
across them.
Chapter 2
Operators
Arithmetic, comparison, logical, bitwise & more

2.1 Arithmetic
a, b = 17, 5
print(a + b) # 22 addition
print(a - b) # 12 subtraction
print(a * b) # 85 multiplication
print(a / b) # 3.4 true division
print(a // b) # 3 floor division
print(a % b) # 2 modulo
print(a ** b) # 1419857 exponentiation

# divmod() gives quotient & remainder together

q, r = divmod(17, 5) # (3, 2)

2.2 Comparison & Chaining

x = 10
print(x == 10) # True
print(x != 5) # True
print(x > 5) # True
print(x < 20) # True
print(x >= 10) # True
print(x <= 10) # True

# Chained comparisons (Python unique feature)

print(1 < 2 < 3) # True
print(0 <= x <= 20) # True
print(1 < 2 > 0 < 3) # True

2.3 Logical
print(True and False) # False
print(True or False) # True
print(not True) # False
 # Short-circuit: second operand skipped if result is decided
x = None
name = x or "Default" # "Default"
y = [] or {} or 0 or "fallback" # "fallback"

# and returns last evaluated, or returns first truthy

print(1 and 2) # 2
print(0 or 3) # 3

2.4 Bitwise
a, b = 0b1010, 0b1100 # 10 and 12
print(bin(a & b)) # 0b1000 AND
print(bin(a | b)) # 0b1110 OR
print(bin(a ^ b)) # 0b0110 XOR
print(bin(~a)) # -0b1011 NOT
print(bin(a << 2)) # 0b101000 left shift (x4)
print(bin(a >> 1)) # 0b101 right shift (÷2)

# Power of 2 check
def is_power_of_2(n): return n > 0 and (n & (n-1)) == 0
print(is_power_of_2(8), is_power_of_2(6)) # True False

2.5 Assignment, Identity, Membership

# Augmented assignment
x = 10; x += 5; x -= 3; x *= 2; x //= 4; x **= 2

# Walrus operator := (3.8+)

import re
data = "user@[Link]"
if m := [Link](r"(.+)@(.+)", data):
print([Link](1), [Link](2)) # user [Link]

# Identity
a = [1,2]; b = a; c = [1,2]
print(a is b) # True — same object
print(a is c) # False — equal value, different object
# Membership
print("py" in "python") # True
print(3 in [1,2,3]) # True
print("x" not in {"a","b"}) # True

✏ Practice Exercises
8. Use divmod() to convert 1000 minutes into hours and minutes.
9. Write is_even() and is_odd() using only the bitwise & operator.
10. Demonstrate the walrus operator in a while loop that reads non-empty user input.
11. Build a simple bit-flag permission system: READ=1, WRITE=2, EXEC=4. Set, check,
and revoke flags.
12. Show 3 examples where Python short-circuit evaluation prevents a runtime error.
13. Write a chained comparison that validates a score is between 0 and 100 inclusive in
one expression.
Chapter 3
Control Flow
if/elif/else, for, while, comprehensions & match

3.1 if / elif / else

score = 78
if score >= 90:
grade = "A"
elif score >= 80:
grade = "B"
elif score >= 70:
grade = "C"
else:
grade = "F"

# Ternary
status = "pass" if score >= 60 else "fail"

# Nested ternary (use sparingly)

label = "high" if score>80 else "mid" if score>60 else "low"

3.2 match / case (Python 3.10+)

Structural pattern matching — far more powerful than a switch statement.
command = ("move", 10, 20)

match command:
case ("quit"):
print("quitting")
case ("move", x, y):
print(f"move to ({x},{y})")
case ("fire", direction) if direction in ("N","S","E","W"):
print(f"fire {direction}")
case _:
print("unknown command")

# Match on types
def process(value):
 match value:
case int() | float() as n if n > 0:
return f"positive number: {n}"
case str() as s if [Link]("err"):
return f"error string: {s}"
case list() as lst:
return f"list with {len(lst)} items"
case _:
return "unknown"

3.3 for Loops

# Iterable
for char in "python": print(char, end=" ")

# range(start, stop, step)

for i in range(0,10,2): print(i,end=" ") # 0 2 4 6 8

# enumerate
names = ["Alice","Bob","Carol"]
for i, name in enumerate(names, start=1):
print(f"{i}. {name}")

# zip
scores = [85,92,78]
for name,score in zip(names,scores):
print(f"{name}: {score}")

# zip_longest
from itertools import zip_longest
for a,b in zip_longest([1,2,3],[4,5],fillvalue=0):
print(a,b)

# for…else (else runs if no break)

for n in range(2,10):
for f in range(2,n):
if n%f == 0: break
else:
print(n,"is prime")
3.4 while, break, continue, pass
# while
count = 0
while count < 5:
if count == 3: count+=1; continue
print(count); count+=1

# Sentinel pattern
while (line := input("> ")) != "quit":
print("Got:", line)

# pass — placeholder body

class EmptyClass: pass
def not_yet_implemented(): pass

✏ Practice Exercises
14. FizzBuzz 1–100: Fizz÷3, Buzz÷5, FizzBuzz÷15, else the number.
15. Write a menu-driven calculator using match/case for operations.
16. Print a 10×10 multiplication table using nested for loops.
17. Write a prime sieve (Sieve of Eratosthenes) up to N using loops and lists.
18. Use for…else to search a list for a target — print "found" or "not found" without a flag
variable.
19. Collect user input lines until a blank line, then print sorted unique words and their
frequencies.
20. Write a number guessing game (1–100) with hints and a 7-guess limit.
Chapter 4
Functions
Parameters, defaults, *args, **kwargs, closures

4.1 Defining & Calling

def greet(name, greeting="Hello"):
"""Return a greeting string."""
return f"{greeting}, {name}!"

print(greet("Alice")) # Hello, Alice!

print(greet("Bob", "Hi")) # Hi, Bob!
print(greet(greeting="Hey","Carol")) # positional before keyword

4.2 *args, **kwargs & Keyword-Only

def total(*amounts): # *args → tuple
return sum(amounts)

def profile(**info): # **kwargs → dict

for k,v in [Link]():
print(f" {k}: {v}")

# Keyword-only (after *)
def resize(width, height, *, scale=1.0, dpi=96):
return int(width*scale*dpi), int(height*scale*dpi)

# Positional-only (before /)
def circle_area(r, /):
import math; return [Link] * r**2

# Full signature
def full(pos_only, /, normal, *, kw_only="a", **extra):
pass

4.3 Return Values & Multiple Returns

def min_max(lst):
 return min(lst), max(lst) # returns tuple

lo, hi = min_max([3,1,7,2,9])

# Early return
def safe_div(a, b):
if b == 0: return None
return a / b

4.4 Scope & LEGB

x = "global"

def outer():
x = "enclosing"
def inner():
x = "local"
print(x) # local
inner()
print(x) # enclosing

outer()
print(x) # global

# global / nonlocal
counter = 0
def inc():
global counter
counter += 1

def make_counter():
n = 0
def bump():
nonlocal n; n += 1; return n
return bump
c = make_counter()
print(c(), c(), c()) # 1 2 3

4.5 Closures
 def multiplier(factor):
def multiply(n): # captures factor
return n * factor
return multiply

double = multiplier(2)
triple = multiplier(3)
print(double(5), triple(5)) # 10 15

# Closure cells
print(double.__code__.co_freevars) # ('factor',)
print(double.__closure__[0].cell_contents) # 2

4.6 Decorators
from functools import wraps
import time

def timer(func):
@wraps(func)
def wrapper(*args, **kwargs):
t = time.perf_counter()
result = func(*args, **kwargs)
print(f"{func.__name__}: {time.perf_counter()-t:.4f}s")
return result
return wrapper

# Decorator factory (decorator with arguments)

def retry(times=3, exceptions=(Exception,)):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(1, times+1):
try:
return func(*args, **kwargs)
except exceptions as e:
if attempt == times: raise
print(f"Retry {attempt}/{times}: {e}")
return wrapper
return decorator

@timer
 @retry(times=3, exceptions=(ValueError,))
def risky(x):
if x < 0: raise ValueError("negative")
return x**2

4.7 functools Utilities

from functools import lru_cache, partial, reduce

@lru_cache(maxsize=None)
def fib(n):
if n < 2: return n
return fib(n-1) + fib(n-2)

# partial — fix arguments

def power(base, exp): return base ** exp
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5), cube(3)) # 25 27

# reduce
from functools import reduce
product = reduce(lambda acc,x: acc*x, [1,2,3,4,5]) # 120

✏ Practice Exercises
21. Write a memoize decorator from scratch (without functools). It should cache results by
args.
22. Write a @validate decorator that checks all numeric arguments are positive before
calling the function.
23. Implement partial() from scratch using closures.
24. Write a function pipeline(*funcs) that returns a function applying funcs left-to-right.
25. Create a @once decorator that ensures a function runs only the first time and returns
cached result afterwards.
26. Build a curried add function: add(1)(2)(3) returns 6.
27. Write a @deprecated(msg) decorator that prints a DeprecationWarning then calls the
function.
Chapter 5
Classes & OOP
__init__, inheritance, polymorphism, dunder methods

5.1 Defining Classes

class Dog:
species = "Canis lupus familiaris" # class attribute

def __init__(self, name, age):

[Link] = name # instance attribute
[Link] = age

def bark(self):
return f"{[Link]}: Woof!"

def __repr__(self):
return f"Dog({[Link]!r}, {[Link]})"

def __str__(self):
return f"{[Link]} (age {[Link]})"

rex = Dog("Rex", 3)
print(repr(rex)) # Dog('Rex', 3)
print(str(rex)) # Rex (age 3)

5.2 Inheritance & super()

class Animal:
def __init__(self, name):
[Link] = name
def speak(self): raise NotImplementedError

class Cat(Animal):
def speak(self): return f"{[Link]}: Meow!"

class Dog(Animal):
def __init__(self, name, breed):
super().__init__(name)
 [Link] = breed
def speak(self): return f"{[Link]}: Woof!"

# Polymorphism
for a in [Cat("Whiskers"), Dog("Rex","Lab")]:
print([Link]())

# Multiple inheritance
class Flyable:
def fly(self): return "I can fly"

class FlyingCat(Cat, Flyable): pass # MRO: FlyingCat→Cat→Animal→Flyable

print(FlyingCat.__mro__)

5.3 Dunder (Magic) Methods

class Vector:
def __init__(self, x, y): self.x,self.y = x,y
def __add__(self, o): return Vector(self.x+o.x, self.y+o.y)
def __sub__(self, o): return Vector(self.x-o.x, self.y-o.y)
def __mul__(self, s): return Vector(self.x*s, self.y*s)
def __rmul__(self, s): return self.__mul__(s)
def __neg__(self): return Vector(-self.x, -self.y)
def __abs__(self): return (self.x**2+self.y**2)**0.5
def __eq__(self, o): return self.x==o.x and self.y==o.y
def __lt__(self, o): return abs(self) < abs(o)
def __len__(self): return 2
def __iter__(self): return iter((self.x, self.y))
def __getitem__(self,i): return (self.x,self.y)[i]
def __repr__(self): return f"Vector({self.x},{self.y})"

v1, v2 = Vector(1,2), Vector(3,4)

print(v1+v2) # Vector(4,6)
print(abs(v2)) # 5.0
print(*v1) # 1 2

5.4 @property, @classmethod, @staticmethod

class Temperature:
def __init__(self, celsius=0):
self._c = celsius
 @property
def celsius(self): return self._c

@[Link]
def celsius(self, val):
if val < -273.15: raise ValueError("Below absolute zero")
self._c = val

@property
def fahrenheit(self): return self._c * 9/5 + 32

@[Link]
def fahrenheit(self, val): self._c = (val-32)*5/9

@property
def kelvin(self): return self._c + 273.15

@classmethod
def from_fahrenheit(cls, f): return cls((f-32)*5/9)

@staticmethod
def is_valid(c): return c >= -273.15

def __repr__(self): return f"Temperature({self._c}°C)"

t = Temperature(100)
print([Link]) # 212.0
t2 = Temperature.from_fahrenheit(32)
print([Link]) # 0.0

✏ Practice Exercises
28. Build a BankAccount class with deposit, withdraw (raises InsufficientFunds), balance
property, and transaction history.
29. Create a Shape hierarchy: Shape (abstract) → Circle, Rectangle, Triangle with area(),
perimeter(), __gt__, __lt__.
30. Implement a Fraction class with full arithmetic (+,-,*,/) and comparison operators.
Always reduce to lowest terms.
31. Write a Matrix class: __add__, __mul__ (matrix & scalar), __getitem__, __setitem__,
transpose(), __repr__.
32. Create a LinkedList class with append, prepend, remove, __iter__, __len__,
 __contains__, __repr__.
33. Implement a mixin class Serializable that adds to_json() and from_json() to any class
using __dict__.
34. Design a role-based Employee hierarchy: Employee → Manager, Engineer.
Manager.add_report() / reports property.
Chapter 6
Modules & Packages
import, packages, __init__.py, pip, virtual envs

6.1 Importing
import math
import math as m # alias
from math import pi, sqrt # specific names
from math import * # all — avoid in production

print([Link], [Link](16), pi)

# Conditional import
try:
import ujson as json
except ImportError:
import json

6.2 Writing Your Own Module

# file: [Link]
PI = 3.14159

def circle_area(r):
return PI * r ** 2

def is_prime(n):
if n < 2: return False
return all(n % i for i in range(2, int(n**0.5)+1))

if __name__ == "__main__":
# Only runs when called directly, not when imported
print(circle_area(5))

6.3 Packages
# mypkg/
 # __init__.py
# shapes/
# __init__.py
# [Link]
# [Link]

# mypkg/__init__.py
from .[Link] import Circle
from .[Link] import Rectangle
__version__ = "1.0.0"
__all__ = ["Circle","Rectangle"] # limits "from pkg import *"

# Usage
from mypkg import Circle, Rectangle
from [Link] import Circle # explicit

6.4 pip & Virtual Environments

# Create & activate venv
python -m venv .venv
source .venv/bin/activate # Linux/Mac
# .venv\Scripts\activate # Windows

# pip commands
# pip install requests
# pip install requests==2.31.0
# pip install "requests>=2.0,<3.0"
# pip install -e . (editable install)
# pip install -r [Link]
# pip freeze > [Link]
# pip list --outdated
# pip show requests

💡 Module search order

Python searches: (1) current directory, (2) PYTHONPATH dirs, (3) standard library, (4) site-
packages. Use [Link] to inspect and modify.

✏ Practice Exercises
35. Create a string_utils package with modules: [Link] (camel, snake, title converters),
 [Link] (word_count, char_freq), [Link] (is_email, is_url).
36. Add a __version__, __author__, and __all__ to your package __init__.py. Verify "from
pkg import *" only exposes __all__.
37. Write a script that uses [Link] to temporarily add a directory, import a module from
it, then restore [Link].
38. Use importlib.import_module() to dynamically load a plugin based on a user-supplied
string.
39. Set up a virtual environment, install requests and beautifulsoup4, freeze
requirements, then recreate from the file.
Chapter 7
File I/O & Context Managers
open(), read/write, json, csv, pathlib

7.1 Reading & Writing Files

# Read entire file
with open("[Link]", "r", encoding="utf-8") as f:
content = [Link]()

# Read line by line (memory-efficient)

with open("[Link]") as f:
for line in f:
print([Link]())

# Write (overwrites)
with open("[Link]", "w") as f:
[Link]("Hello\n")
[Link](["line1\n","line2\n"])

# Append
with open("[Link]", "a") as f:
[Link]("new entry\n")

# Binary
with open("[Link]","rb") as f:
data = [Link]()

7.2 pathlib — Modern Path Handling

from pathlib import Path

p = Path("mydir") / "sub" / "[Link]"

[Link](parents=True, exist_ok=True)
p.write_text("content", encoding="utf-8")
print(p.read_text())

# Querying
print([Link](), p.is_file(), [Link])
 for f in Path(".").glob("**/*.py"):
print(f, [Link]().st_size, "bytes")

7.3 JSON & CSV

import json, csv

# JSON
data = {"name":"Alice","scores":[90,85]}
json_str = [Link](data, indent=2)
back = [Link](json_str)
Path("[Link]").write_text(json_str)

# CSV
rows = [["Name","Age"],["Alice",30],["Bob",25]]
with open("[Link]","w",newline="") as f:
[Link](f).writerows(rows)

with open("[Link]") as f:
for row in [Link](f):
print(row["Name"], row["Age"])

7.4 Custom Context Managers

# Class-based
class ManagedDB:
def __init__(self, url): [Link] = url
def __enter__(self):
print(f"Connecting to {[Link]}")
[Link] = "connection_object"
return [Link]
def __exit__(self, exc_type, exc_val, exc_tb):
print("Disconnecting")
return False # do not suppress exceptions

# Generator-based (simpler)
from contextlib import contextmanager

@contextmanager
def timer(label=""):
import time
 start = time.perf_counter()
try:
yield
finally:
elapsed = time.perf_counter()-start
print(f"{label}: {elapsed:.4f}s")

with timer("my task"):

sum(range(1_000_000))

✏ Practice Exercises
40. Write a program using pathlib to find all .py files recursively, count lines in each, print
a sorted summary.
41. Build a JSON-based contact book: add, list, search by name, delete. Persist between
runs.
42. Write a context manager @atomic_write(path) that writes to a temp file then renames
— guaranteeing no partial writes.
43. Parse a CSV of transactions (date, amount, category) — compute monthly totals and
write a summary CSV.
44. Implement a rotating log file context manager that creates a new file when the current
one exceeds 1 MB.
 PART II
Advanced Python
Type hints, async, concurrency, metaclasses
Chapter 8
Error Handling
try/except/finally, custom exceptions, exception chains

8.1 try / except / else / finally

try:
x = int(input("Enter number: "))
result = 100 / x
except ValueError as e:
print(f"Bad input: {e}")
except ZeroDivisionError:
print("Cannot divide by zero")
except (TypeError, OverflowError) as e:
print(f"Math error: {e}")
else:
print(f"Result: {result}") # only if no exception
finally:
print("Cleanup — always runs")

8.2 Exception Hierarchy

# BaseException
# SystemExit, KeyboardInterrupt, GeneratorExit
# Exception
# ArithmeticError: ZeroDivisionError, OverflowError
# LookupError: KeyError, IndexError
# ValueError, TypeError, AttributeError
# OSError: FileNotFoundError, PermissionError
# RuntimeError: RecursionError
# StopIteration

# Catch broad then narrow — or narrow first

try:
risky_operation()
except FileNotFoundError: # specific first
handle_missing_file()
except OSError: # broader catches the rest
handle_os_error()
8.3 Custom Exceptions
class AppError(Exception):
"""Base for all app errors."""

class ValidationError(AppError):
def __init__(self, field, message, value=None):
[Link] = field
[Link] = value
super().__init__(f"[{field}] {message} (got {value!r})")

class DatabaseError(AppError):
pass

class RecordNotFoundError(DatabaseError):
def __init__(self, model, pk):
super().__init__(f"{model} with pk={pk} not found")
[Link], [Link] = model, pk

# Raising with context (exception chaining)

def get_user(user_id):
try:
return [Link](user_id)
except DBConnectionError as e:
raise DatabaseError("Cannot fetch user") from e # __cause__

# Suppress chaining
raise DatabaseError("cannot fetch") from None # hides chain

8.4 Exception Groups (Python 3.11+)

# ExceptionGroup — multiple exceptions at once
try:
raise ExceptionGroup("multiple errors", [
ValueError("bad value"),
TypeError("wrong type"),
])
except* ValueError as eg:
print("ValueErrors:", [Link])
except* TypeError as eg:
 print("TypeErrors:", [Link])

8.5 Context Managers for Cleanup

from contextlib import suppress, ExitStack

# suppress — silently ignore specific exceptions

with suppress(FileNotFoundError):
Path("[Link]").unlink()

# ExitStack — dynamic context managers

with ExitStack() as stack:
files = [stack.enter_context(open(f)) for f in file_list]
# all files auto-closed on exit

✏ Practice Exercises
45. Write a safe_json_load(path) that returns {} on FileNotFoundError, None on
JSONDecodeError, and re-raises others.
46. Create a full custom exception hierarchy for a REST API: APIError → AuthError,
NotFoundError, RateLimitError.
47. Write a @catch_and_log(logger, default=None) decorator that catches Exception,
logs it, and returns default.
48. Implement try_parse(s, type_) that returns (value, None) on success, (None, error) on
failure (Result pattern).
49. Build a multi-step form validator that collects all validation errors before raising an
ExceptionGroup.
50. Write a function that demonstrates exception chaining (from e) vs suppression (from
None) with print output.
Chapter 9
Comprehensions, Generators & Lambda
List/dict/set comprehensions, generator expressions, functional tools

9.1 List Comprehensions

squares = [x**2 for x in range(10)]
evens = [x for x in range(20) if x%2==0]
flat = [x for row in [[1,2],[3,4]] for x in row]
nested = [[i*j for j in range(1,4)] for i in range(1,4)]

# Walrus in comprehension
data = [1,-2,3,-4,5]
pos_doubled = [y for x in data if (y:=x*2) > 0]
print(pos_doubled) # [2, 6, 10]

9.2 Dict & Set Comprehensions

words = ["hello","world","python"]
lengths = {w: len(w) for w in words}
uniq_len = {len(w) for w in words}
inverted = {v:k for k,v in {"a":1,"b":2}.items()}

# Filter while building

adults = {name: age for name,age in [Link]() if age>=18}

9.3 Generator Expressions

# Lazy — one value at a time, no list built in memory
total = sum(x**2 for x in range(10**7)) # fast, low mem

# Chained generators — pipeline

lines = ([Link]() for line in open("[Link]"))
nonempty = (l for l in lines if l)
words = (w for l in nonempty for w in [Link]())
counts = {}
for w in words: counts[w] = [Link](w,0)+1
9.4 lambda, map, filter, reduce
double = lambda x: x*2
add = lambda a,b: a+b

# map / filter return lazy iterators

nums = [1,2,3,4,5]
doubled = list(map(lambda x: x*2, nums))
odds = list(filter(lambda x: x%2, nums))

# sorted with key

people = [{"name":"Bob","age":30},{"name":"Alice","age":25}]
[Link](key=lambda p: p["age"])

# reduce
from functools import reduce
fact = reduce(lambda acc,x: acc*x, range(1,6)) # 120

# operator module (faster than lambda)

from operator import mul, attrgetter
fact2 = reduce(mul, range(1,6))

✏ Practice Exercises
51. Use a single list comprehension to extract all email addresses from a list of strings.
52. Build a comprehension-only word frequency counter from a paragraph of text.
53. Write a generator pipeline: read CSV → parse → filter rows where amount>100 →
yield formatted strings.
54. Recreate map(), filter(), and zip() as generator functions without importing anything.
55. Generate all Pythagorean triples (a,b,c) where a,b,c ≤ 50 using a list comprehension.
56. Use reduce() to flatten a list of lists into one list.
Chapter 10
Iterators & Generators
Iterator protocol, yield, send(), yield from, async generators

10.1 Iterator Protocol

class CountUp:
def __init__(self, start, stop, step=1):
[Link], [Link], [Link] = start, stop, step

def __iter__(self): return self

def __next__(self):
if [Link] > [Link]: raise StopIteration
val = [Link]
[Link] += [Link]
return val

# Behind the scenes of a for loop

it = iter([1,2,3])
while True:
try: print(next(it))
except StopIteration: break

10.2 Generator Functions (yield)

def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a+b

fib = fibonacci()
print([next(fib) for _ in range(10)])
# [0,1,1,2,3,5,8,13,21,34]

# Generator with cleanup

def managed():
print("setup")
 try:
yield 1
yield 2
finally:
print("cleanup") # runs even on .close()

10.3 yield from

def chain(*iterables):
for it in iterables:
yield from it

def flatten(nested):
for item in nested:
if isinstance(item,(list,tuple)):
yield from flatten(item)
else:
yield item

print(list(flatten([1,[2,[3,4]],[5]]))) # [1,2,3,4,5]

10.4 send() & throw() — Coroutines

def accumulator():
total = 0
while True:
value = yield total
if value is None: return
total += value

gen = accumulator()
next(gen) # prime
print([Link](10)) # 10
print([Link](20)) # 30
print([Link](5)) # 35

# throw() — inject exception

[Link](GeneratorExit)
✏ Practice Exercises
57. Write a generator range2d(rows, cols) that yields (row, col) tuples for a 2D grid.
58. Implement [Link](), islice(), and takewhile() as generator functions.
59. Build a generator-based event system: a dispatcher yields events, handlers use
send() to acknowledge.
60. Write a recursive tree traversal (pre-order, in-order, post-order) as generators using
yield from.
61. Create a buffered generator that reads a file in chunks of N bytes, yielding one line at
a time.
62. Implement a coroutine pipeline: producer → transformer → consumer using send()
and next().
Chapter 11
Standard Library
os, sys, json, datetime, collections, itertools & more

11.1 os & pathlib

import os
from pathlib import Path

# Environment
print([Link]("HOME","/tmp"))
[Link]["MY_VAR"] = "hello"

# File system
[Link]("a/b/c", exist_ok=True)
[Link]("[Link]","[Link]")

# Walk tree
for root,dirs,files in [Link]("."):
for f in files:
full = [Link](root,f)
print(full, [Link](full))

# pathlib (modern)
p = Path("data") / "[Link]"
print([Link], [Link], [Link])
matches = list(Path(".").rglob("*.py"))

11.2 sys
import sys
print([Link], [Link])
print([Link]) # CLI args
[Link](0,"./mods") # add to import path
print([Link]()) # 1000 default
[Link](2000)

# stdin/stdout/stderr
for line in [Link]:
 [Link]([Link]())

# Exit codes
[Link](0) # 0 = success, nonzero = error

11.3 collections
from collections import
Counter,defaultdict,deque,namedtuple,OrderedDict,ChainMap

# Counter
words = "the quick brown fox jumps over the lazy dog".split()
c = Counter(words)
print(c.most_common(3)) # [('the',2),...]
[Link](["the","fox"])
print(+c) # elements with count>0

# defaultdict
dd = defaultdict(list)
for name,grade in [("Alice",90),("Bob",85),("Alice",88)]:
dd[name].append(grade)
# {"Alice":[90,88],"Bob":[85]}

# deque — O(1) both ends

dq = deque([1,2,3], maxlen=5)
[Link](0); [Link](4)
[Link](1) # shift right

# namedtuple
Point = namedtuple("Point",["x","y"])
p = Point(3,4); print(p.x, p._asdict())

# ChainMap — layered dicts

defaults = {"color":"blue","size":10}
overrides = {"color":"red"}
merged = ChainMap(overrides, defaults)
print(merged["color"],merged["size"]) # red 10

11.4 itertools
import itertools as it
 # Infinite
count = [Link](start=0, step=2) # 0,2,4,6...
cycle = [Link]("ABC") # A,B,C,A,B,C...
repeat = [Link](10, times=3) # 10,10,10

# Finite
print(list([Link]([1,2],[3,4],[5]))) # [1,2,3,4,5]
print(list([Link]("ABCDE",[1,0,1,0,1])))# [A,C,E]
print(list([Link](lambda x:x<3,[1,2,3,4])))
print(list([Link](lambda x:x<3,[1,2,3,4])))
print(list([Link](count, 5))) # [0,2,4,6,8]

# Combinatoric
print(list([Link]("AB",2))) # [(A,B),(B,A)]
print(list([Link]("ABC",2))) # [(A,B),(A,C),(B,C)]
print(list([Link]("AB","12"))) # (A1)(A2)(B1)(B2)

# groupby (input must be sorted)

data = sorted([("A",1),("A",2),("B",3)],key=lambda x:x[0])
for key,grp in [Link](data,key=lambda x:x[0]):
print(key, list(grp))

11.5 datetime & time

from datetime import datetime, date, timedelta, timezone
import time

now = [Link]()
utc = [Link]([Link])
today = [Link]()

# Format
print([Link]("%Y-%m-%d %H:%M:%S"))
dt = [Link]("2024-01-15",""%Y-%m-%d")

# Arithmetic
tomorrow = today + timedelta(days=1)
week_ago = now - timedelta(weeks=1)
diff = datetime(2026,1,1) - now
print([Link],"days until 2026")
 # Unix timestamp
ts = [Link]() # seconds since epoch
dt2 = [Link](ts)

11.6 re — Regular Expressions

import re

pattern = r"\b[A-Z][a-z]+\b"
text = "Alice and Bob met Carol."

# Search / match / findall / finditer

print([Link](pattern, text)) # ['Alice','Bob','Carol']

m = [Link](r"(\d{4})-(\d{2})-(\d{2})", "2024-01-15")
if m:
year,month,day = [Link]()

# Sub / subn
clean = [Link](r"\s+", " ", "too many spaces")

# Compiled pattern (reuse)

email_re = [Link](r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[A-Z|a-z]{2,}")
emails = email_re.findall(text)

# Verbose mode
phone = [Link](r"""
(\d{3}) # area code
[-.] # separator
(\d{3}) # prefix
[-.] # separator
(\d{4}) # line number
""", [Link])

✏ Practice Exercises
63. Use Counter + most_common() to build a word frequency report for a text file, sorted
by frequency.
64. Use defaultdict(set) to build an inverted index: {word → set of line numbers} from a
file.
65. Build all 5-card poker hands (no repeats) from a 52-card deck using
[Link]. Count flush hands.
66. Use re to validate and parse email addresses, phone numbers, and ISO dates from a
block of text.
67. Write a CLI tool using [Link] that accepts --sort, --filter=PATTERN flags to process
a CSV file.
68. Use groupby to aggregate a list of sales records by month, computing total and
average per month.
Chapter 12
Type Hints & Static Typing
PEP 484, mypy, generics, Protocol, TypeVar

Type hints make Python code self-documenting and catch bugs before runtime. They are purely
advisory at runtime — use mypy or pyright to enforce them.

12.1 Basic Annotations

def greet(name: str, times: int = 1) -> str:
return (f"Hello, {name}!\n") * times

age: int = 30
labels: list[str] = ["a","b","c"] # Python 3.9+
mapping: dict[str, int] = {"x":1}
pair: tuple[int, str] = (1,"a") # fixed-length tuple
coords: tuple[float, ...] = (1.0,2.0,3.0) # variable length

12.2 Optional, Union, Any

from typing import Optional, Union, Any

# Optional[X] ≡ Union[X, None]

def find(lst: list[int], val: int) -> Optional[int]:
try: return [Link](val)
except ValueError: return None

# Union — multiple types (Python 3.10+ use |)

def process(value: int | str) -> str:
return str(value)

# Any — opt out of type checking

def dynamic(x: Any) -> Any:
return x

12.3 Generics & TypeVar

from typing import TypeVar, Generic
 T = TypeVar("T")
K = TypeVar("K")
V = TypeVar("V")

def first(lst: list[T]) -> T:

return lst[0]

class Stack(Generic[T]):
def __init__(self) -> None:
self._items: list[T] = []

def push(self, item: T) -> None:

self._items.append(item)

def pop(self) -> T:

return self._items.pop()

def peek(self) -> T:

return self._items[-1]

s: Stack[int] = Stack()
[Link](1); [Link](2)
print([Link]()) # 2

12.4 Protocol — Structural Subtyping

from typing import Protocol, runtime_checkable

@runtime_checkable
class Drawable(Protocol):
def draw(self) -> None: ...
def area(self) -> float: ...

class Circle:
def __init__(self, r: float): self.r = r
def draw(self) -> None: print("O")
def area(self) -> float: return 3.14*self.r**2

class Square:
def __init__(self, s: float): self.s = s
def draw(self) -> None: print("[]")
 def area(self) -> float: return self.s**2

# Neither inherits Drawable — structural match is enough

def render(shape: Drawable) -> None:
[Link]()
print([Link]())

render(Circle(5)) # works
render(Square(4)) # works
print(isinstance(Circle(1), Drawable)) # True — runtime_checkable

12.5 Callable, Literal, Final, TypedDict

from typing import Callable, Literal, Final, TypedDict

# Callable[[arg_types...], return_type]
def apply(func: Callable[[int], int], x: int) -> int:
return func(x)

# Literal — restricted set of values

Direction = Literal["N","S","E","W"]
def move(d: Direction, steps: int) -> None: ...

# Final — cannot be reassigned

MAX: Final = 100

# TypedDict — typed dict structure

class Movie(TypedDict):
title: str
year: int
rating: float

m: Movie = {"title":"Inception","year":2010,"rating":8.8}

🔧 Running mypy
pip install mypy then mypy your_file.py or mypy --strict your_file.py. For gradual adoption,
annotate public APIs first and use # type: ignore sparingly.

✏ Practice Exercises
69. Take Chapter 5's BankAccount class and add full type annotations. Run mypy --strict
on it.
70. Write a typed generic class Queue[T] with enqueue, dequeue, peek, is_empty,
__len__.
71. Define a Protocol Comparable with __lt__ and __eq__. Write a generic typed_sort(lst:
list[C]) function.
72. Annotate a function that accepts a callback: transform(data: list[T], fn: Callable[[T], U])
-> list[U].
73. Create a TypedDict for a User and write load_users(path: Path) -> list[User] with full
type annotations.
Chapter 13
ABCs & Dataclasses
abc module, @dataclass, NamedTuple, field()

13.1 Abstract Base Classes

from abc import ABC, abstractmethod

class Shape(ABC):
@abstractmethod
def area(self) -> float: ...

@abstractmethod
def perimeter(self) -> float: ...

def describe(self) -> str: # concrete method

return f"{type(self).__name__}: area={[Link]():.2f}"

class Circle(Shape):
def __init__(self, r: float): self.r = r
def area(self) -> float: return 3.14159 * self.r**2
def perimeter(self) -> float: return 2 * 3.14159 * self.r

# Cannot instantiate Shape directly

# Shape() → TypeError: Can't instantiate abstract class

13.2 Registering Virtual Subclasses

from abc import ABC

class JSONSerializable(ABC):
@abstractmethod
def to_json(self) -> str: ...

# Register a class without inheriting

import json
@[Link]
class MyDict(dict):
def to_json(self): return [Link](self)
 print(isinstance(MyDict(), JSONSerializable)) # True

13.3 @dataclass
from dataclasses import dataclass, field, asdict, astuple

@dataclass
class Point:
x: float
y: float = 0.0

@dataclass(order=True, frozen=True) # frozen = immutable

class Product:
sort_index: float = field(init=False, repr=False)
name: str
price: float
tags: list[str] = field(default_factory=list)

def __post_init__(self):
object.__setattr__(self, "sort_index", [Link])

p1 = Product("Widget", 9.99, ["sale"])

p2 = Product("Gadget", 19.99)
print(p1 < p2) # True (sorted by sort_index/price)
print(asdict(p1)) # {'name':'Widget','price':9.99,...}

13.4 Typed NamedTuple

from typing import NamedTuple

class Coordinate(NamedTuple):
lat: float
lon: float
altitude: float = 0.0

def distance_to(self, other: "Coordinate") -> float:

return (([Link])**2 + ([Link])**2)**0.5

nyc = Coordinate(40.7128, -74.0060)

la = Coordinate(34.0522, -118.2437)
print(nyc.distance_to(la))
print(nyc._asdict()) # OrderedDict

✏ Practice Exercises
74. Create an abstract Animal hierarchy with abstract speak(), move(), and concrete
describe(). Implement 4 subclasses.
75. Build a @dataclass Config with validation in __post_init__ that ensures all string fields
are non-empty.
76. Use @dataclass(frozen=True) for a Money class. Implement __add__, __mul__.
Money objects should be hashable.
77. Create a NamedTuple Version(major, minor, patch) with __str__ returning "v1.2.3"
and __lt__ for comparisons.
78. Write an abstract Repository[T] ABC with get, save, delete, list_all. Implement
InMemoryRepository[T].
Chapter 14
Metaclasses & Descriptors
__slots__, descriptors, __init_subclass__, metaclasses

These are the advanced OOP tools used to build frameworks, ORMs, and APIs in Python.
Django's Model and SQLAlchemy's Column both use these internally.

14.1 __slots__
class WithDict:
def __init__(self, x, y):
self.x, self.y = x, y

class WithSlots:
__slots__ = ("x","y") # no __dict__, fixed attributes
def __init__(self, x, y):
self.x, self.y = x, y

import sys
a = WithDict(1,2); b = WithSlots(1,2)
print([Link](a.__dict__)) # ~200 bytes
# b has no __dict__ — ~50-60 bytes, faster attribute access

# Inheritance with __slots__

class Point3D(WithSlots):
__slots__ = ("z",) # only add new slots
def __init__(self, x, y, z):
super().__init__(x,y)
self.z = z

14.2 Descriptors
class Validator:
"""Non-data descriptor (read-only)."""
def __set_name__(self, owner, name):
[Link] = name

def __get__(self, obj, objtype=None):

if obj is None: return self
return obj.__dict__.get([Link])
 class PositiveNumber:
"""Data descriptor (read + write)."""
def __set_name__(self, owner, name):
[Link] = f"_{name}"

def __get__(self, obj, objtype=None):

if obj is None: return self
return getattr(obj, [Link], None)

def __set__(self, obj, value):

if not isinstance(value,(int,float)) or value<=0:
raise ValueError(f"{[Link]} must be positive")
setattr(obj, [Link], value)

class Circle:
radius = PositiveNumber()
def __init__(self, r): [Link] = r

c = Circle(5)
print([Link]) # 5
# Circle(-1) → ValueError

14.3 __init_subclass__
class Plugin:
registry: dict[str,type] = {}

def __init_subclass__(cls, name: str, **kwargs):

super().__init_subclass__(**kwargs)
[Link][name] = cls
cls.plugin_name = name

class JsonPlugin(Plugin, name="json"):

def process(self, data): return str(data)

class CsvPlugin(Plugin, name="csv"):

def process(self, data): return ",".join(map(str,data))

print([Link])
# {"json":<class JsonPlugin>, "csv":<class CsvPlugin>}
14.4 Metaclasses
# type is the metaclass of all classes
print(type(int)) # <class 'type'>
print(type(list)) # <class 'type'>

# Custom metaclass
class Singleton(type):
_instances: dict = {}

def __call__(cls, *args, **kwargs):

if cls not in cls._instances:
cls._instances[cls] = super().__call__(*args,**kwargs)
return cls._instances[cls]

class Database(metaclass=Singleton):
def __init__(self): [Link] = False

db1 = Database()
db2 = Database()
print(db1 is db2) # True

# Metaclass that auto-registers methods

class AutoLogger(type):
def __new__(mcs, name, bases, namespace):
for attr,val in [Link]():
if callable(val) and not [Link]("_"):
namespace[attr] = mcs._wrap(val)
return super().__new__(mcs,name,bases,namespace)

@staticmethod
def _wrap(func):
from functools import wraps
@wraps(func)
def wrapper(*a,**kw):
print(f"Calling {func.__name__}")
return func(*a,**kw)
return wrapper
✏ Practice Exercises
79. Rewrite the BankAccount class using __slots__. Benchmark memory usage vs a
regular class for 1,000,000 instances.
80. Write a TypeEnforced descriptor that validates assigned values against a specified
type.
81. Use __init_subclass__ to build a command registry: any subclass of Command auto-
registers itself by name.
82. Implement a Singleton metaclass. Verify with tests that only one instance is ever
created per class.
83. Build a simple ORM-style Row metaclass: class User(Row): id=IntField();
name=StrField() auto-maps to dict.
Chapter 15
Async / Await & asyncio
Coroutines, event loop, tasks, streams, semaphores

asyncio enables concurrent I/O without threads. It is the foundation of async web frameworks
(FastAPI, aiohttp), async database clients, and high-performance servers.

15.1 Coroutines & async def

import asyncio

async def fetch(url: str) -> str:

await [Link](0.1) # non-blocking pause
return f"data from {url}"

async def main():

result = await fetch("[Link]
print(result)

[Link](main())

15.2 Tasks & gather()

import asyncio, time

async def download(n: int) -> str:

await [Link](1) # simulate I/O
return f"file_{n}"

async def main():

start = time.perf_counter()

# Sequential — slow: 3 seconds total

for i in range(3):
await download(i)

# Concurrent — fast: ~1 second total

results = await [Link](
download(0), download(1), download(2)
 )
print(results)
print(f"Elapsed: {time.perf_counter()-start:.2f}s")

[Link](main())

15.3 Task management

async def main():
# create_task — fires immediately, no await yet
t1 = asyncio.create_task(download(1), name="d1")
t2 = asyncio.create_task(download(2), name="d2")

await [Link](0) # yield control

# Cancel a task
[Link]()
try:
await t2
except [Link]:
print("t2 was cancelled")

# Wait with timeout

try:
result = await asyncio.wait_for(download(3), timeout=0.5)
except [Link]:
print("timed out")

15.4 Semaphores, Queues & Locks

import asyncio

async def rate_limited_fetch(sem, url):

async with sem: # max N concurrent
await [Link](0.1)
return f"data:{url}"

async def main():

sem = [Link](5) # max 5 concurrent
urls = [f"url/{i}" for i in range(20)]
 results = await [Link](
*(rate_limited_fetch(sem, u) for u in urls)
)

# [Link] — producer/consumer
async def producer(q):
for i in range(5):
await [Link](i)
await [Link](0.1)
await [Link](None) # sentinel

async def consumer(q):

while True:
item = await [Link]()
if item is None: break
print(f"Got: {item}")
q.task_done()

15.5 Async Generators & Context Managers

import asyncio

# Async generator
async def ticker(delay, to):
for i in range(to):
yield i
await [Link](delay)

async def main():

async for i in ticker(0.1, 5):
print(i)

# Async context manager

class AsyncDB:
async def __aenter__(self):
await [Link](0.01) # connect
return self
async def __aexit__(self, *args):
await [Link](0.01) # disconnect

async def query():

 async with AsyncDB() as db:
pass # use db

⚡ When to use asyncio

Use async when your bottleneck is I/O (network, disk, DB). For CPU-bound tasks, use
multiprocessing instead — asyncio does not bypass the GIL.

✏ Practice Exercises
84. Write an async function that fetches 10 URLs concurrently using [Link]() with
[Link]() as mock.
85. Implement an async producer-consumer queue where 3 producers and 2 consumers
run concurrently.
86. Write an async retry decorator: retries the coroutine up to N times on exception with
exponential backoff.
87. Create an async context manager that logs the start/end time of any async block.
88. Build an async rate limiter: allows at most N calls per second using
[Link] and [Link].
Chapter 16
Concurrency — Threading, Multiprocessing
& the GIL
When to use which, GIL explained, [Link]

16.1 The GIL Explained

The Global Interpreter Lock (GIL) is a mutex in CPython that allows only one thread to execute
Python bytecode at a time. This means threading does NOT speed up CPU-bound work — but
it does not affect I/O-bound work because threads release the GIL while waiting for I/O.
# GIL rule of thumb:
# CPU-bound → multiprocessing (bypasses GIL)
# I/O-bound → threading or asyncio
# Mixed → [Link] (abstracts both)

16.2 threading
import threading, time

def worker(name, delay):

print(f"{name} starting")
[Link](delay)
print(f"{name} done")

threads = [[Link](target=worker,args=(f"T{i}",1)) for i in

range(4)]
for t in threads: [Link]()
for t in threads: [Link]()

# Thread-safe counter
lock = [Link]()
counter = 0

def inc():
global counter
for _ in range(100_000):
with lock:
counter += 1
 ts = [[Link](target=inc) for _ in range(4)]
for t in ts: [Link]()
for t in ts: [Link]()
print(counter) # 400000

16.3 multiprocessing
from multiprocessing import Process, Pool, Queue, Manager
import os

def cpu_task(n):
return sum(i*i for i in range(n))

# Pool — parallel map

with Pool(processes=4) as pool:
results = [Link](cpu_task, [10**6]*8)
print(sum(results))

# Shared state
def counter_worker(shared_count, lock):
for _ in range(1000):
with lock:
shared_count.value += 1

from multiprocessing import Value

import ctypes
count = Value(ctypes.c_int, 0)

16.4 [Link] — Unified Interface

from [Link] import ThreadPoolExecutor, ProcessPoolExecutor,
as_completed
import requests # pip install requests

urls = [f"[Link] for i in range(8)]

# ThreadPoolExecutor — for I/O bound

with ThreadPoolExecutor(max_workers=4) as ex:
futures = {[Link]([Link], url): url for url in urls}
for future in as_completed(futures):
url = futures[future]
 try:
resp = [Link](timeout=5)
print(url, resp.status_code)
except Exception as e:
print(url, "failed:", e)

# ProcessPoolExecutor — for CPU bound

def fib(n):
if n<2: return n
return fib(n-1)+fib(n-2)

with ProcessPoolExecutor() as ex:

results = list([Link](fib, range(30,36)))

✏ Practice Exercises
89. Write a program that demonstrates the GIL: two CPU-bound threads vs two
processes — time both.
90. Build a thread-safe LRU cache using [Link] and [Link].
91. Implement a parallel file processor: use ProcessPoolExecutor to apply a
transformation to every file in a directory.
92. Create a producer-consumer pipeline using [Link] with 2 producers and 3
consumers.
93. Write an I/O-bound benchmark: download 20 URLs using (a) sequential, (b)
threading, (c) asyncio — compare times.
 PART III
Professional Python
Memory, testing, packaging, patterns & projects
Chapter 17
Memory Model & Garbage Collection
Reference counting, cyclic GC, weakref, __del__

17.1 How Python Manages Memory

Every Python object has a reference count. When it drops to zero the object is immediately
deallocated. This handles most cases. CPython also has a cyclic garbage collector for reference
cycles.
import sys, gc

# Reference counting
x = [1, 2, 3]
print([Link](x)) # 2 (x + argument to getrefcount)

y = x # refcount → 3
del y # refcount → 2
del x # refcount → 0 → freed

# Object size
print([Link]([])) # ~56 bytes
print([Link]([1]*1000)) # ~8056 bytes

# Cyclic GC
import gc
[Link]() # manual collection
print(gc.get_threshold()) # (700, 10, 10) — generation thresholds
[Link]() # disable (use with caution)

17.2 Reference Cycles

# A → B → A creates a cycle — refcounts never hit 0
class Node:
def __init__(self, val): [Link] = val; [Link] = None

a = Node(1)
b = Node(2)
[Link] = b
[Link] = a # cycle — leaks without gc
 del a, b
[Link]() # cyclic gc detects and frees them

17.3 weakref
import weakref

class Cache:
def __init__(self): self._store = {}

def set(self, key, obj):

self._store[key] = [Link](obj) # weak reference

def get(self, key):

ref = self._store.get(key)
return ref() if ref else None # ref() returns obj or None

class BigData:
def __init__(self, n): [Link] = list(range(n))

cache = Cache()
data = BigData(1000)
[Link]("big", data)
print([Link]("big")) # <BigData ...>
del data
print([Link]("big")) # None — object freed

17.4 __del__ & Object Lifecycle

class Resource:
def __init__(self, name):
[Link] = name
print(f" {name}: acquired")

def __del__(self):
print(f" {[Link]}: released")

r = Resource("DB")
del r # immediately: "DB: released"
# context manager is better than __del__ for cleanup
from contextlib import contextmanager
@contextmanager
def resource(name):
print(f"{name}: acquired")
try: yield
finally: print(f"{name}: released")

✏ Practice Exercises
94. Write a memory_usage() function using [Link]() that deep-counts nested
containers.
95. Create a circular reference with two objects. Confirm [Link]() frees them by
tracking object count.
96. Build a WeakValueDictionary cache that automatically evicts entries when objects are
garbage collected.
97. Profile memory usage of a @dataclass vs a regular class vs __slots__ for 1,000,000
instances.
98. Implement an object pool using weakref that reuses objects instead of creating new
ones.
Chapter 18
Testing with pytest
pytest, fixtures, parametrize, mocking, coverage

Writing tests is as important as writing code. pytest is the de-facto Python testing framework —
expressive, extensible, and powerful.

18.1 pytest Basics

# test_math.py
def add(a, b): return a + b
def divide(a, b):
if b == 0: raise ZeroDivisionError
return a / b

# prefix function with test_

def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
assert add(0, 0) == 0

def test_divide():
assert divide(10, 2) == 5.0

def test_divide_by_zero():
import pytest
with [Link](ZeroDivisionError):
divide(1, 0)

# Run: pytest test_math.py -v

18.2 Fixtures
import pytest
from pathlib import Path

@[Link]
def sample_data():
return [1, 2, 3, 4, 5]
 @[Link]
def tmp_file(tmp_path):
f = tmp_path / "[Link]"
f.write_text("hello")
return f

def test_sum(sample_data):
assert sum(sample_data) == 15

def test_file_content(tmp_file):
assert tmp_file.read_text() == "hello"

# Scoped fixtures
@[Link](scope="module") # created once per module
def db_connection():
conn = create_connection()
yield conn
[Link]()

18.3 Parametrize
import pytest

@[Link]("a,b,expected", [
(1, 2, 3),
(-1, 1, 0),
(0, 0, 0),
(100, -50, 50),
(0.1, 0.2, [Link](0.3)), # float comparison
("hello", " world", "hello world"),
([1,2], [3,4], [1,2,3,4]),
(True, False, 1),
])
def test_add_parametrized(a, b, expected):
assert add(a, b) == expected

# Multiple decorators
@[Link]("n", [1,2,3])
@[Link]("multiplier", [2,3])
def test_multiply(n, multiplier):
 assert n * multiplier == n * multiplier

18.4 Mocking
from [Link] import Mock, MagicMock, patch, call

# Basic mock
m = Mock()
m.some_method.return_value = 42
print(m.some_method()) # 42
m.some_method.assert_called_once()

# patch — replace real objects in tests

def send_email(to, body):
import smtplib
# real implementation
pass

def notify_user(user_id):
send_email(f"{user_id}@[Link]","Welcome!")

def test_notify():
with patch("__main__.send_email") as mock_email:
notify_user(42)
mock_email.assert_called_once_with("42@[Link]","Welcome!")

# patch as decorator
@patch("[Link]")
def test_api_call(mock_get):
mock_get.return_value.json.return_value = {"status":"ok"}
result = call_api("[Link]
assert result["status"] == "ok"

18.5 Coverage & Markers

# Run with coverage
# pytest --cov=mypackage --cov-report=html tests/

# Custom markers
@[Link]
@[Link]
def test_database_roundtrip(): ...

# [Link] or [Link]
# [pytest]
# markers =
# slow: marks tests as slow
# integration: requires external services

# Skip / xfail
@[Link](reason="WIP")
def test_not_ready(): ...

@[Link](reason="known bug #123")

def test_known_bug():
assert broken_function() == 42

✏ Practice Exercises
99. Write a full test suite for a Stack class: push, pop, peek, is_empty, overflow handling.
100. Use @[Link] to test a Caesar cipher with 10 different inputs
and expected outputs.
101. Write a test that mocks the file system: test a function that reads, processes,
and writes a file.
102. Create fixtures for a temporary SQLite database. Write 5 tests that use it with
scope="function".
103. Write a property-based test using the hypothesis library: test that
reverse(reverse(lst)) == lst for all lists.
Chapter 19
Packaging & Distribution
[Link], setuptools, building & publishing to PyPI

19.1 Project Structure

# Modern Python project layout
# mypackage/
# src/
# mypackage/
# __init__.py
# [Link]
# [Link]
# tests/
# [Link]
# test_core.py
# [Link]
# [Link]
# LICENSE

19.2 [Link]
# [Link]
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "[Link]:build"

[project]
name = "mypackage"
version = "1.0.0"
description = "A great Python package"
readme = "[Link]"
license = {file = "LICENSE"}
requires-python = ">=3.11"
keywords = ["example","package"]

authors = [
{name = "Alice Smith", email = "alice@[Link]"},
]
 dependencies = [
"requests>=2.31",
"click>=8.0",
]

[[Link]-dependencies]
dev = ["pytest","mypy","ruff"]

[[Link]]
my-tool = "[Link]:main" # creates CLI command

[[Link]]
strict = true

[[Link]]
line-length = 88

19.3 Building & Publishing

# 1. Install build tools
# pip install build twine

# 2. Build distributions
# python -m build
# → dist/
# [Link]
# [Link]

# 3. Check the build

# twine check dist/*

# 4. Upload to TestPyPI first

# twine upload --repository testpypi dist/*

# 5. Install from TestPyPI

# pip install --index-url [Link] mypackage

# 6. Upload to real PyPI

# twine upload dist/*
19.4 Versioning (SemVer)
# [Link]
# 1.0.0 — stable release
# 1.1.0 — new feature (backward-compatible)
# 1.1.1 — bug fix
# 2.0.0 — breaking change

# Using bumpversion / bump2version

# bump2version minor → 1.0.0 → 1.1.0

# Dynamic version from git tags (setuptools-scm)

# [tool.setuptools_dynamic]
# version = {attr = "mypackage.__version__"}

✏ Practice Exercises
104. Create a full Python package with [Link], src layout, and a CLI entry
point using click.
105. Add mypy and ruff to your package's dev dependencies and create a Makefile
with lint, type-check, test targets.
106. Write a [Link] for your package using the Keep a Changelog
format.
107. Set up GitHub Actions CI: run tests on Python 3.11 and 3.12 on push and PR.
108. Publish your package to TestPyPI and install it in a fresh virtual environment
to verify it works.
Chapter 20
Design Patterns in Python
Pythonic implementations of classic patterns

Classic GoF patterns often look very different in Python — many are simplified by first-class
functions, decorators, and duck typing.

20.1 Creational Patterns

Singleton
class Singleton:
_instance = None

def __new__(cls):
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance

# Pythonic: module-level variable (modules are singletons)

# [Link] → import config; config.db_url = "..."

Factory
from abc import ABC, abstractmethod

class Serializer(ABC):
@abstractmethod
def serialize(self, data: dict) -> str: ...

class JsonSerializer(Serializer):
def serialize(self, data): import json; return [Link](data)

class CsvSerializer(Serializer):
def serialize(self, data):
return ",".join(str(v) for v in [Link]())

def get_serializer(fmt: str) -> Serializer:

registry = {"json": JsonSerializer, "csv": CsvSerializer}
cls = [Link](fmt)
if not cls: raise ValueError(f"Unknown format: {fmt}")
 return cls()

Builder
class QueryBuilder:
def __init__(self): self._parts = {}

def table(self, name):

self._parts["table"] = name; return self

def select(self, *cols):

self._parts["select"] = cols or ("*",); return self

def where(self, condition):

self._parts.setdefault("where",[]).append(condition); return self

def limit(self, n):

self._parts["limit"] = n; return self

def build(self):
t = self._parts.get("table","?")
s = ",".join(self._parts.get("select",("*",)))
w = " AND ".join(self._parts.get("where",[]))
l = self._parts.get("limit","")
q = f"SELECT {s} FROM {t}"
if w: q += f" WHERE {w}"
if l: q += f" LIMIT {l}"
return q

q = (QueryBuilder().table("users").select("id","name")
.where("age > 18").where("active = 1").limit(10).build())
print(q)

20.2 Structural Patterns

Decorator Pattern (not to be confused with Python decorator syntax)
from abc import ABC, abstractmethod

class Coffee(ABC):
@abstractmethod
 def cost(self) -> float: ...
@abstractmethod
def description(self) -> str: ...

class SimpleCoffee(Coffee):
def cost(self) -> float: return 1.0
def description(self) -> str: return "Coffee"

class MilkDecorator(Coffee):
def __init__(self, coffee: Coffee): self._c = coffee
def cost(self) -> float: return self._c.cost() + 0.5
def description(self) -> str: return self._c.description() + ", Milk"

class SugarDecorator(Coffee):
def __init__(self, coffee: Coffee): self._c = coffee
def cost(self) -> float: return self._c.cost() + 0.25
def description(self) -> str: return self._c.description() + ", Sugar"

c = SugarDecorator(MilkDecorator(SimpleCoffee()))
print([Link](), [Link]()) # Coffee, Milk, Sugar 1.75

Proxy & Adapter

# Adapter: wrap an incompatible interface
class OldAPI:
def get_data(self, id): return {"id":id}

class NewAPIAdapter:
def __init__(self, old): self._old = old
def fetch(self, resource_id): return self._old.get_data(resource_id)

# Proxy: control access

class SecureProxy:
def __init__(self, real_obj, allowed_roles):
self._obj = real_obj
self._roles = allowed_roles

def __getattr__(self, name):

import functools
attr = getattr(self._obj, name)
if not callable(attr): return attr
@[Link](attr)
 def check_access(*a,**kw):
if current_user_role() not in self._roles:
raise PermissionError(f"Access denied to {name}")
return attr(*a,**kw)
return check_access

20.3 Behavioural Patterns

Observer (Event System)
from collections import defaultdict
from typing import Callable

class EventEmitter:
def __init__(self):
self._handlers: dict[str,list[Callable]] = defaultdict(list)

def on(self, event: str, handler: Callable):

self._handlers[event].append(handler)
return self

def off(self, event: str, handler: Callable):

self._handlers[event].remove(handler)

def emit(self, event: str, *args, **kwargs):

for h in self._handlers[event]:
h(*args, **kwargs)

emitter = EventEmitter()
[Link]("login", lambda user: print(f"Welcome {user}!"))
[Link]("login", lambda user: log_event("login", user))
[Link]("login", "Alice")

Strategy
from typing import Callable

# Pythonic: strategy is just a callable

def sort_data(data: list, strategy: Callable = sorted) -> list:
return strategy(data)
 # Strategies
asc = sorted
desc = lambda d: sorted(d, reverse=True)
by_length = lambda d: sorted(d, key=len)

names = ["Charlie","Alice","Bob"]
print(sort_data(names, asc))
print(sort_data(names, by_length))

Command Pattern
from dataclasses import dataclass, field
from typing import Protocol

class Command(Protocol):
def execute(self) -> None: ...
def undo(self) -> None: ...

@dataclass
class WriteText:
doc: list
text: str
_prev: str = field(init=False)

def execute(self):
self._prev = [Link][-1] if [Link] else ""
[Link]([Link])

def undo(self):
[Link]()

class Editor:
def __init__(self):
[Link], [Link] = [], []

def run(self, cmd: Command):

[Link]()
[Link](cmd)

def undo(self):
if [Link]:
[Link]().undo()
✏ Practice Exercises
109. Implement a plugin system using the Factory pattern and __init_subclass__
auto-registration.
110. Build an event-driven order system: Order emits
"placed","shipped","delivered" events; handlers log and notify.
111. Implement undo/redo for a text editor using the Command pattern. Support at
least 10 commands.
112. Write a caching Proxy for any callable that stores results by argument hash.
113. Create a Builder for constructing HTTP requests with method, url, headers,
query params, body fluent API.
Chapter 21
Performance & Profiling
timeit, cProfile, line_profiler, tips & tricks

21.1 Measuring Performance

import timeit

# timeit — microbenchmark
t1 = [Link]("sum(range(1000))", number=10000)
t2 = [Link]("[x for x in range(1000)]", number=10000)
print(f"sum: {t1:.3f}s list: {t2:.3f}s")

# time.perf_counter — wall clock

import time
start = time.perf_counter()
result = expensive()
print(f"{time.perf_counter()-start:.6f}s")

21.2 cProfile
import cProfile, pstats, io

pr = [Link]()
[Link]()
# ... code to profile ...
[Link]()

s = [Link]()
ps = [Link](pr, stream=s).sort_stats("cumulative")
ps.print_stats(20)
print([Link]())

# Command line
# python -m cProfile -o [Link] [Link]
# python -m pstats [Link]

21.3 Key Optimisation Techniques

 # 1. Use built-ins — they're implemented in C
total = sum(lst) # not: t=0; for x in lst: t+=x
result = any(pred(x) for x in lst) # short-circuits

# 2. List/dict/set comprehensions > loops

squares = [x*x for x in range(n)] # ~2x faster than append loop

# 3. Local variables faster than global

from math import sqrt as _sqrt # bind to local

# 4. Generator over list for large data

total = sum(x**2 for x in range(10**8)) # no list allocation

# 5. [Link]() not +=
parts = ["a","b","c","d"]
result = "".join(parts) # not: r=""; for p in parts: r+=p

# 6. defaultdict / Counter vs manual dict

from collections import Counter
freq = Counter(words) # not: for w in words: d[w]=[Link](w,0)+1

# 7. numpy for numeric arrays

import numpy as np
a = [Link](lst)
print([Link](), [Link](), [Link]()) # vectorised, no Python loop

21.4 Memory Optimisation

# Use generators instead of lists
lines = ([Link]() for l in open("[Link]")) # no full load

# __slots__ for many small objects (see Ch 14)

# array module for numeric arrays (leaner than list)

import array
arr = [Link]("i", range(1000)) # C int array — 4 bytes each
print(arr.buffer_info(), [Link])

# mmap for large files

import mmap
with open("[Link]","rb") as f:
 mm = [Link]([Link](), 0, access=mmap.ACCESS_READ)
# read without loading entire file into memory

✏ Practice Exercises
114. Profile a function that builds a frequency table three ways: dict loop,
defaultdict, Counter. Show timings.
115. Benchmark string concatenation (+= vs join) for 10,000 and 1,000,000 strings.
116. Use cProfile to find the bottleneck in a recursive Fibonacci implementation.
Optimise with lru_cache.
117. Compare memory usage of list vs generator vs array for 1,000,000 floats
using [Link] and tracemalloc.
118. Write a benchmark comparing Python dict lookup vs list search at various
sizes — explain the Big-O difference.
Chapter 22
Advanced Standard Library
logging, argparse, contextlib, functools, operator

22.1 logging
import logging

# Basic config
[Link](
level=[Link],
format="%(asctime)s %(name)s %(levelname)s %(message)s",
datefmt="%H:%M:%S",
handlers=[
[Link](),
[Link]("[Link]"),
]
)

log = [Link](__name__)

[Link]("detail")
[Link]("normal operation")
[Link]("unexpected but handled")
[Link]("something failed")
[Link]("system cannot continue")

# Structured logging
[Link]("User logged in", extra={"user_id":42, "ip":"[Link]"})

# Log exceptions
try:
1/0
except ZeroDivisionError:
[Link]("Math error") # includes traceback

22.2 argparse
import argparse, sys
 def build_parser():
parser = [Link](
description="Process CSV files",
formatter_class=[Link],
)
parser.add_argument("input", type=str, help="Input CSV path")
parser.add_argument("-o","--output", default="[Link]")
parser.add_argument("-n","--limit", type=int, default=None)
parser.add_argument("--verbose", action="store_true")
parser.add_argument("--format", choices=["csv","json","tsv"])
parser.add_argument("--columns", nargs="+")
return parser

def main():
args = build_parser().parse_args()
if [Link]:
[Link](level=[Link])
process([Link], [Link], [Link])

if __name__ == "__main__":
main()

22.3 contextlib extras

from contextlib import (
contextmanager, asynccontextmanager,
suppress, ExitStack, redirect_stdout,
nullcontext,
AbstractContextManager,
)

# redirect_stdout — capture print output

import io
buf = [Link]()
with redirect_stdout(buf):
print("captured")
print([Link]()) # "captured\n"

# nullcontext — optional context manager

def process(data, lock=None):
ctx = lock or nullcontext()
 with ctx:
do_work(data)

# ExitStack — dynamic number of context managers

with ExitStack() as stack:
handles = [stack.enter_context(open(f)) for f in files]
# All closed at end, even on exception

22.4 functools & operator

from functools import (
lru_cache, cache, # memoization
partial, partialmethod, # partial application
reduce, accumulate, # fold operations
singledispatch, # function overloading
total_ordering, # auto-generate comparison methods
)

from operator import (

add, mul, sub, truediv, # arithmetic
itemgetter, attrgetter, # key functions
methodcaller, # call method by name
)

# total_ordering: define __eq__ + __lt__, get the rest free

from functools import total_ordering
@total_ordering
class Card:
RANKS = "23456789TJQKA"
def __init__(self, rank): [Link] = rank
def __eq__(self, o): return [Link] == [Link]
def __lt__(self, o): return [Link]([Link]) <
[Link]([Link])

# singledispatch — function overloading by type

@singledispatch
def process(data):
raise TypeError(f"Unsupported type: {type(data)}")

@[Link](int)
def _(n): return n * 2
@[Link](str)
def _(s): return [Link]()

@[Link](list)
def _(lst): return [process(x) for x in lst]

✏ Practice Exercises
119. Build a logging setup with two handlers: DEBUG to file, WARNING+ to
console. Use RotatingFileHandler.
120. Write a CLI tool with argparse: subcommands (add, remove, list) for a todo list
stored in JSON.
121. Use singledispatch to write a serialize() function that handles int, float, str, list,
dict differently.
122. Use total_ordering on a Version(major,minor,patch) class and test all 6
comparison operators.
123. Write a retry context manager using ExitStack that retries the block N times on
specified exceptions.
Chapter 23
Advanced Patterns & Idioms
Context variables, __class_getitem__, slots+descriptors combo

23.1 contextvars — Context-local State

import contextvars, asyncio

request_id: [Link][str] = [Link](

"request_id", default="none"
)

async def handle_request(rid: str):

token = request_id.set(rid)
try:
await do_work()
finally:
request_id.reset(token)

async def do_work():

print(f"[{request_id.get()}] doing work")

async def main():

await [Link](
handle_request("req-1"),
handle_request("req-2"),
)

23.2 __class_getitem__ — Generic Classes

class TypedList:
"""A list that enforces element type."""
def __class_getitem__(cls, item_type):
class _TypedList(list):
_type = item_type
def append(self, v):
if not isinstance(v, self._type):
raise TypeError(f"Expected {self._type.__name__}")
super().append(v)
 _TypedList.__name__ = f"TypedList[{item_type.__name__}]"
return _TypedList

IntList = TypedList[int]
lst = IntList()
[Link](1) # OK
[Link]("x") # TypeError

23.3 __init_subclass__ for Plugin Systems

class Exporter:
_registry: dict[str, type] = {}

def __init_subclass__(cls, fmt: str = "", **kw):

super().__init_subclass__(**kw)
if fmt:
Exporter._registry[fmt] = cls

@classmethod
def for_format(cls, fmt: str) -> "Exporter":
klass = cls._registry.get(fmt)
if not klass: raise ValueError(f"No exporter for {fmt!r}")
return klass()

def export(self, data): raise NotImplementedError

class JsonExporter(Exporter, fmt="json"):

def export(self, data): import json; return [Link](data)

class CsvExporter(Exporter, fmt="csv"):

def export(self, data):
return "\n".join(",".join(str(v) for v in row) for row in data)

e = Exporter.for_format("json")
print([Link]({"a":1}))

23.4 Advanced Descriptor: Validated Field

from typing import TypeVar, Generic, Any, Type
T = TypeVar("T")

class Field(Generic[T]):
"""Descriptor that validates type on assignment."""

def __init__(self, typ: Type[T], *, default: Any = None, nullable=True):

self._type = typ
self._default = default
self._nullable = nullable

def __set_name__(self, owner, name):

[Link] = name
[Link] = f"_{name}"

def __get__(self, obj, objtype=None):

if obj is None: return self
return getattr(obj, [Link], self._default)

def __set__(self, obj, value):

if value is None and self._nullable:
setattr(obj, [Link], value); return
if not isinstance(value, self._type):
raise TypeError(f"{[Link]} must be {self._type.__name__},
got {type(value).__name__}")
setattr(obj, [Link], value)

class User:
name = Field(str, nullable=False)
age = Field(int, default=0)
email = Field(str)

def __init__(self, name, age=0, email=None):

[Link], [Link], [Link] = name, age, email

✏ Practice Exercises
124. Use contextvars to implement a request-scoped logger that prefixes all
messages with the current request ID.
125. Build a validated Schema system using descriptors: IntField, StrField,
FloatField with min/max constraints.
126. Implement a generic Pair[T, U] class using __class_getitem__ that supports
iteration and unpacking.
127. Create a full plugin architecture with Exporter/Importer base classes, auto-
registration, and a CLI frontend.
Chapter 24
Capstone Projects
Full projects combining all concepts

These four projects are designed to be completed over several days each. They combine
concepts from all three parts of the book.

Project 1: Async Web Scraper & Analyser

Difficulty: Advanced | Concepts: asyncio, aiohttp, dataclasses, type hints, generators, csv/json,
logging, argparse, testing
# Features to build:
# - async fetch() with retry + backoff using asyncio
# - Semaphore-controlled concurrency (max N simultaneous)
# - HTML parsing with [Link] (stdlib)
# - Data models using @dataclass and TypedDict
# - Generator pipeline: urls → fetch → parse → clean → save
# - Output to CSV and JSON
# - CLI with argparse: --urls FILE --concurrency N --output DIR
# - Logging with RotatingFileHandler
# - Full pytest suite with mocked HTTP responses

Project 2: Task Queue & Worker System

Difficulty: Advanced | Concepts: multiprocessing, threading, queues, abc, type hints, logging,
design patterns
# Features to build:
# - Task abstract base class with execute(), on_success(), on_failure()
# - Worker pool using ProcessPoolExecutor
# - Priority queue (heapq) for task scheduling
# - Retry logic with exponential backoff
# - Task status tracking: PENDING → RUNNING → DONE/FAILED
# - In-memory result store with weakref cache
# - Observer pattern for task lifecycle events
# - CLI monitor showing active workers and queue depth

Project 3: Typed ORM (Mini SQLAlchemy)

Difficulty: Expert | Concepts: metaclasses, descriptors, type hints, generics, sqlite3, testing
 # Features to build:
# - Field descriptors: IntField, StrField, FloatField, BoolField
# - ModelMeta metaclass that collects fields and table name
# - Model base class with save(), delete(), __repr__
# - Query builder: [Link]().where().order_by().limit()
# - Relationships: ForeignKey descriptor with lazy loading
# - Migration tracking (simple version table)
# - Full test suite with in-memory SQLite

# Example usage target:

# class User(Model, table="users"):
# id = IntField(primary_key=True)
# name = StrField(max_length=100)
# age = IntField(nullable=True)

# user = User(name="Alice", age=30)

# [Link]()
# users = [Link]().where("age > 18").order_by("name").all()

Project 4: Data Pipeline Framework

Difficulty: Advanced | Concepts: generics, protocols, generators, typing, contextlib, logging,
packaging
# Features to build:
# - Source protocol: iter_records() -> Iterator[Record]
# - Transform protocol: transform(record) -> Record | None
# - Sink protocol: write(record) -> None
# - Pipeline class that chains source → [transforms] → sink
# - Built-in sources: CSVSource, JSONSource, SQLiteSource
# - Built-in transforms: FilterTransform, MapTransform, BatchTransform
# - Built-in sinks: CSVSink, JSONSink, ConsoleSink
# - Stats: records_in, records_out, errors, elapsed time
# - @pipeline decorator to compose pipelines declaratively

# Target usage:
# @pipeline
# def etl(source=CSVSource("[Link]"), sink=JSONSink("[Link]")):
# return (
# Filter(lambda r: r["age"] > 18),
# Map(lambda r: {**r, "name": r["name"].title()}),
# )
What to do next
You have now covered everything from basic variables to metaclasses, async programming,
testing, and design patterns. Here is a suggested path forward:
• Web Development: FastAPI (async REST APIs), Django (full-stack), SQLAlchemy (ORM)
• Data Science: numpy, pandas, matplotlib, scikit-learn, Jupyter
• AI & ML: PyTorch, TensorFlow, Hugging Face transformers, LangChain
• DevOps: Docker, GitHub Actions CI/CD, Poetry, pre-commit hooks
• Systems: Cython (C extensions), ctypes, cffi, CFFI for interfacing with C libraries

Happy Coding! 🐍