Til posts by Daniel Roy Greenfeld

TIL: ^ bitwise XOR

2025-05-07T03:30:43.170061+00:00

The bitwise XOR operator ^, also known as exclusive or, can be used to compare boolean objects to see if one and only one is True.

Let's see it in action, first comparing three False booleans, which will return False.

```python {.marimo} False ^ False ^ False

Now let's demonstrate three different combinations of a single `True` and any number of `False` booleans, which will return `True` in each case.

```python {.marimo}
print(True ^ False ^ False)
print(False ^ True ^ False)
print(False ^ False ^ True)

However, if we have two or more values that are True, the result will be False because more than one value are True.

```python {.marimo} print(True ^ True ^ False) print(True ^ False ^ True) print(False ^ True ^ True)

## What about non-boolean types?

The `^` operator only works with boolean types. If you try to use it on non-boolean types, you'll get a `TypeError`. For example, if you try to use it on integers or strings to check for truthiness, you'll get an error.

```python
'' ^ '' ^ 'one'

>>>
TypeError                   Traceback (most recent call last)
Cell In[20], line 1
>>> 1 '' ^ '' ^ 'one'

TypeError: unsupported operand type(s) for ^: 'str' and 'str'

To make this comparison works, you can convert the non-boolean types to boolean first. For example, you can use the bool() function to convert an integer or string to a boolean before using the ^ operator.

```python {.marimo} bool('') ^ bool('') ^ bool('one')

## Updates

- 2025-05-07: Updated the post to describe XOR only returns `True` if one (and only one) of the values is `True`. Removed the segment on `any()` as it is too far away in design from `XOR`.  Credit for this fix goes to [Curt Merrill](https://bsky.app/profile/cmerrill.com) and [Rens Dimmendaal](https://rensdimmendaal.com/).

```python {.marimo hide_code="true"}
mo.md(
    r"""

    """
)

python {.marimo} import marimo as mo

TIL: Exception.add_note

2025-05-13T14:10:31.105025+00:00

Python 3.11 introduced a new method called add_note for exceptions, which allows you to add extra information to exceptions in an easy, intuitive way.

try:
    1/0
except ZeroDivisionError as e:
    e.add_note("This is a note about the error")
    e.add_note("This is another note")
    e.add_note("All notes must be strings")
    raise

The output will look something like this:

=========================================================
ZeroDivisionError       Traceback (most recent call last)
Cell In[1], line 2
      1 try:
====> 2     1/0
      3 except ZeroDivisionError as e:
      4     e.add_note("This is a note about the error")

ZeroDivisionError: division by zero
This is a note about the error
This is another note
All notes must be strings

See those last three lines? Those are the notes we added to the exception!

The add_note allows for adding extra info without overriding the normal error, printing after the rest of the exception. The notes must be strings, and are stored in a list at the __notes__ attribute.

Finally, technically speaking, add_note is a method of the BaseException class, which is the base class for Python exceptions. This means that we can use it with any and all Python exceptions.

TIL: HTML 404 errors for FastAPI

2025-06-13T04:23:23.230032+00:00

I had a TIL about custom responses in FastAPI but there's enough nuance that it deserves a full blog post. In the meantime, here's a TIL about custom HTTP responses.

from fastapi import FastAPI
from fastapi.responses import HTMLResponse


async def custom_404_exception_handler(request, exc):
    return HTMLResponse(
        f'<p>404 Not Found at "{request.url.path}"</p>', status_code=404
    )

# Add more HTTP exceptions as needed
HTTP_EXCEPTIONS = {404: custom_404_exception_handler}

app = FastAPI(exception_handlers=HTTP_EXCEPTIONS)


@app.get("/")
async def read_root():
    return {"Hello": "World"}

Try it out by running the app and going to a non-existent path, like /not-found. You should see a simple HTML page with a 404 message.

TIL: Removing exif geodata from media

2025-06-23T20:56:49.362878+00:00

To do this you'll need exiftool. On Mac it can be installed with brew install exiftool.

How to remove the geodata:

exiftool "-gps*=" beach-video.mp4

TIL: Single source version package builds with uv

2025-07-23T01:59:20.353392+00:00

Remove version in pyproject.toml and replace with dynamic = ["version"]
Add [tool.setuptools.dynamic] and specify the location of the version using this dialogue: version = { attr = "mypackage.__version__" }
In your project's root __init__.py, add a __version__ attribute.

Example:

# pyproject.toml
[project]
name = "mypackage"
dynamic = ["version"]
# version = "0.1.0" # Don't set version here

[tool.setuptools.dynamic]
version = { attr = "mypackage.__version__" } # Set version here

Don't forget to specify the version in Python:

# mypackage/__init__.py
__version__ = "0.1.0"

Thanks to Audrey Roy Greenfeld for pairing with me on getting this to work.

TIL: How to type args and kwargs

2025-07-26T09:15:07.896468+00:00

An oddity of my work for a while has been that I haven't used *args and **kwargs with type annotations. Recently, however I've been working on code that leans on those things a lot. And I've been ignoring setting types there because this fails wretchedly with type checking libraries:

# This fails type checks. :(
from typing import Any

def func(*args: tuple[str], **kwargs: dict[str, Any]):
    pass

That's because the types you set in the args are the value of the container. So to make my code pass the type checkers I need to do:

# This passes type checks!
from typing import Any

def func(*args: str, **kwargs: Any):
    pass

Thanks to Will McGugan for explaining this to me. I'm a fan of his work, which includes the inestimable Rich and Textualize libraries!

Update

2025-07-28: Luke Plant points out that Adam Johnson has a good article on this topic: https://adamj.eu/tech/2021/05/11/python-type-hints-args-and-kwargs/

TIL: Single source version package builds with uv (redux)

2025-08-22T02:20:08.473528+00:00

Not that long ago I wrote about how to use UV to build packages with a single source of truth for the version number. Since then, my friend Adam Johnson has pointed out that I could be doing it better. Here's how he demonstrated I should be doing it instead.

# pyproject.toml
[project]
name = "air"
version = "0.25.0" # This is the source of truth for the version number

The way to check programmatically the version number is to rely not on someone setting __version__ in the code, but rather to use the following technique:

from importlib.metadata import version
version = version("air")
print(version)  # This will print "0.25.0"

Thanks for the tip, Adam! This is a much cleaner and tool friendly way to ensure that the version number is consistent across your package without having to manually update it in multiple places.

TIL: Using SQLModel Asynchronously with FastAPI (and Air) with PostgreSQL

2025-08-29T05:54:58.034855+00:00

First, let's set up our environment. We'll need to install the necessary packages:

uv venv
uv add "FastAPI[standard]" SQLModel asyncpg psycopg2-binary greenlet

A quick quick guide to these dependencies:

FastAPI: The web framework we'll be using. We install with the [standard] extras to make running in the shell easy
SQLModel: The ORM for interacting with our SQL database
psycopg2-binary: A PostgreSQL adapter for Python, which will be used by SQLModel for synchronous operations
asyncpg: An asynchronous PostgreSQL client library, which will be used by SQLModel for async operations
greenlet: A dependency for SQLAlchemy, which powers SQLModel, to support asynchronous operations.

Yes, we need BOTH synchronous and asynchronous PostgreSQL drivers. SQLModel (and SQLAlchemy) use the synchronous driver for certain operations, even when we are mostly working in an async context.

FastAPI

Here we'll set up a simple FastAPI application that uses SQLModel asynchronously (and synchronously). As far as I can tell, to create tables we need to use a synchronous engine, which explains the need for both drivers and engines. Specifically we'll create (synchronously) a Story model and two asynchronous endpoints: one for retrieving stories and another for creating a new story.

When creating my example database, I used 'drg' as my username and called the database 'my-database'. Adjust the connection strings as needed for your setup.

```python
# main.py
from fastapi import Depends, FastAPI
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
from sqlmodel import SQLModel, select, Field, create_engine
from typing import AsyncGenerator


# Step 0: sync engine for non-web db actions like creating tables
sync_engine = create_engine("postgresql://username@localhost/my-database", echo=True)


# Step 1: Create async engine and session
async_engine = create_async_engine(
    "postgresql+asyncpg://username@localhost/my-database",  # Async connection string
    echo=True,  # Optional: Set to False in production
    future=True,
)

# Step 2: Set up async session
async_session = async_sessionmaker(
    bind=async_engine,
    class_=AsyncSession,
    expire_on_commit=False,
)


# Step 3: Create a dependency to yield an async session
async def _get_async_session() -> AsyncGenerator[AsyncSession, None]:
    async with async_session() as session:
        yield session

# Wrap _get_async_session with Depends for FastAPI to keep type checking happy
get_async_session = Depends(_get_async_session)

# Define a SQL model table
class Story(SQLModel, table=True):
    id: int | None = Field(default=None, primary_key=True)
    slug: str

# Create tables (synchronously, outside of request handling)
SQLModel.metadata.create_all(sync_engine)


# Step 4: Create FastAPI app and endpoints
app = FastAPI()


@app.get("/")
async def index(session: AsyncSession = get_async_session):
    stmt = select(Story)
    stories = await session.exec(stmt)
    return {"stories": [x.slug for x in stories]}


@app.post("/{slug}")
async def create_story(slug: str, session: AsyncSession = get_async_session):
    story = Story(slug=slug)
    session.add(story)  # Don't commit standard db actions
    await session.commit()  # Always await commits
    return {"slug": story.slug}

Start it up with:

fastapi dev main.py

Go to the API docs and try it out: http://localhost:8000/docs

Air

Air is a FastAPI-based framework for building HTML-focused web applications. It builds on top of FastAPI and adds features like easier templating, improved form handling, and more. You can combine it with FastAPI routes and SQLModel to create the web pages for your API application.

In any case, except for single import change, as well as defining the app and endpoints, the code is identical to the FastAPI example above. The main difference is that instead of returning JSON responses, we return HTML using Air's components and layouts. Air can return Jinja-powered responses, web but here we'll use Air's built-in components for simplicity.

First let's set up our environment. We'll need to install the necessary packages:

uv venv
uv add "Air[standard]" SQLModel asyncpg psycopg2-binary greenlet

Air: The FastAPI-powered web framework we'll be using. We install with the [standard] extras to make running in the shell easy
SQLModel: The ORM for interacting with our SQL database
psycopg2-binary: A PostgreSQL adapter for Python, which will be used by SQLModel for synchronous operations
asyncpg: An asynchronous PostgreSQL client library, which will be used by SQLModel for async operations
greenlet: A dependency for SQLAlchemy, which powers SQLModel, to support asynchronous operations.

Now for the code. When creating my example database, I used 'drg' as my username and called the database 'my-database'. Adjust the connection strings as needed for your setup.

# main.py
import air  # air is built on FastAPI
from fastapi import Depends
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
from sqlmodel import SQLModel, select, Field, create_engine
from typing import AsyncGenerator


# Step 0: sync engine for non-web db action
sync_engine = create_engine("postgresql://username@localhost/my-database", echo=True)


# Step 1: Create async engine and session
async_engine = create_async_engine(
    "postgresql+asyncpg://username@localhost/my-database",  # Async connection string
    echo=True,  # Optional: Set to False in production
    future=True,
)

# Step 2: Set up async session
async_session = async_sessionmaker(
    bind=async_engine,
    class_=AsyncSession,
    expire_on_commit=False,
)


# Step 3: Create a dependency to yield an async session
async def _get_async_session() -> AsyncGenerator[AsyncSession, None]:
    async with async_session() as session:
        yield session


get_async_session = Depends(_get_async_session)


class Story(SQLModel, table=True):
    id: int | None = Field(default=None, primary_key=True)
    slug: str


SQLModel.metadata.create_all(sync_engine)

# After this comment, the project diverges from the FastAPI example

# Step 4: Create FastAPI app and endpoints
app = air.Air()


@app.page
async def index(session: AsyncSession = get_async_session):
    stmt = select(Story)
    stories = await session.exec(stmt)
    return air.layouts.mvpcss(
        air.Article(
            air.Ol(*[air.Li(x.slug) for x in stories])
        )
    )


@app.get("/{slug}")
async def create_story(slug: str, session: AsyncSession = get_async_session):
    "Note: This is a GET endpoint for simplicity; in a real app, use POST for creating resources."
    story = Story(slug=slug)
    session.add(story)  # Don't commit standard db actions
    await session.commit()  # Always await commits
    return air.layouts.mvpcss(
        air.H1(story.slug),
        air.P("Story created! ", air.A("Go back", href="/"))
    )

Start it up with:

fastapi dev main.py

Try it out:

http://localhost:8000/first-article - Just clicking this will create a new article with the slug "first-article". In a real app, you'd want to use a POST request for creating resources, but this is just for demonstration.
http://localhost:8000/second-article - Create a second article.
http://localhost:8000 - View the list of articles.

TIL: Setting environment variables for pytest

2025-09-02T02:29:33.084710+00:00

When writing tests in pytest, often there's a need to set environment variables for your tests. Instead of modifying os.environ directly, which can lead to side effects and make tests harder to manage, here's how to do it with the pytest-env package.

First, install the package.

pip install pytest-env # classic but works great
uv add pytest-env # if you're one of us cool kids using uv
uv add pytest-env --group test # if you use a specific test group of dependencies

A lot of people use pytest.ini to configure pytest, but I prefer using pyproject.toml for a more modern approach. Here's how you can set environment variables in pyproject.toml:

[tool.pytest_env]
# Test value for the SUPER_SECRET_KEY environment variable
SUPER_SECRET_KEY = "ABC123"

Then run your tests just like you would normally do, and now the environment variable will be picked up by whatever test or code is looking for it. That's correct, you don't need to do any further configuration.

pytest .

Since a lot of API SDK libraries use environment variables for configuration, this is a great way to ensure your tests run in a controlled environment. Either with mock replies or against a test instance of the service.

TIL: Merging two git projects

2025-09-13T08:49:50.670264+00:00

Of course this task can be done with copy/paste. The challenge there is the loss of git history. All the history of struggles and tribulations are gone. More important is the attribution - unless any and all contributors are made co-authors. But then the volume of attribution isn't accurate, some one who made one tiny change gets as much credit as the leading contributor.

A better way is to merge the repos together while preserving the git history. Let's imagine we want to merge repo #2 into repo #1:

AIR (A new Python web framework). This is in a repo on my laptop called air
AirDocs (Documentation for AIR, getting merged into air). This is in a repo on github located at github.com/feldroy/airdocs

Here's how I did it:

Step 1: Add an airdocs git remote to air

cd air/
git remote add -f airdocs git@github.com:feldroy/airdocs.git

Step 2: Merge AirDocs into AIR

git merge airdocs/main --allow-unrelated-histories

At this point code conflicts may arise, which can be resolved. In our case, we made sure that the repos didn't have overlapping files.

TIL: Loading .env files with uv run

2025-09-28T22:44:18.296914+00:00

We don't need python-dotenv, use uv run with --env-file, and your env vars from .env get loaded.

For example, if we've got a FastAPI or Air project we can run it locally with env vars like:

uv run --env-file .env fastapi dev

You can specific different env files for different environments, like .env.dev, .env.prod, etc.

uv run --env-file .env.dev fastapi dev

All credit goes to Audrey Roy Greenfeld for pointing this out.

TIL: Default code block languages for mkdocs

2025-11-22T12:08:34.618632+00:00

When using Mkdocs with Material, you can set default languages for code blocks in your mkdocs.yml configuration file. This is particularly useful for inline code examples that may not have explicit language tags.

markdown_extensions:
  - pymdownx.highlight:
      default_lang: python

You can see what this looks like in practice with Air's API reference for forms here: feldroy.github.io/air/api/forms/. With this configuration, any code block without a specified language defaults to Python syntax highlighting, making documentation clearer and more consistent.

TIL: Using PygmentsRenderer with mistletoe as a partial

2026-02-22T10:09:16.103693+00:00

For the past 18 months or so on this site, I've been using marked.js for the web and python-markdown for the atom feed. I decided not long ago to switch to using mistletoe so there's one consistent source of truth for markdown rendering.

I also thought that instead of defining a reusable function that both the web and atom feed could use, I would just use Python's functools.partial to create a new function that has the renderer argument set to PygmentsRenderer. Normally I prefer fully defined functions over using partial, but in this case, since it is for my personal site it felt like a good use of partial.

Add the dependencies

uv add mistletoe pygments

Create the partial function

from functools import partial

import mistletoe
from mistletoe.contrib.pygments_renderer import PygmentsRenderer

markdown = partial(mistletoe.markdown, renderer=PygmentsRenderer)

Use the new function

html = markdown(markdown_string)

See it in action

Here's the commits where I implemented this change, at least on the web side: Using pygments for highlighting of code. Before I change the atom feed generation to use this, I'll make sure that it renders nicely on planetpython and other feed aggregators. I'll post that in a seperate TIL when I do figure out how to do that.

TIL: Checkov's Gun

2026-03-09T03:12:43.043418+00:00

Chekhov's gun is a storytelling rule that says:

If you show something important early in a story, it should matter later.

I've used this technique for years but didn't know it had a name until today. It's similar to foreshadowing, and objectively just a synonym of that word. My personal interpretation is that it has to do with specific concrete objects or behaviors rather.

One of my favorite uses of Checkov's Gun is in early cinema as well as some Hong Kong action films where the camera will show a background object early in a dramatic scene. That background object will then be used in some way later in the action. Buster Keaton, Jackie Chan, and Sammo Hung are the grandmasters of this technique.

TIL: Improving µCSS readability on mobile

2026-04-06T02:45:46.176600+00:00

With µCSS on mobile, the typography can be hard to read, and media queries appear to be the right approach to fixing it. Adding this bit of CSS below makes text more easy to read on mobile devices. I'm now using this on both this site and my author site.

@media (max-width: 768px) {
  body {
    font-size: 18px;
    line-height: 1.7;
    padding: 0 14px;
  }

  h1, h2, h3 {
    line-height: 1.25;
  }

  pre, code {
    font-size: 0.9em;
    overflow-x: auto;
  }
}