Support numeric constraints for decimal.Decimal by Siyet · Pull Request #1006 · msgspec/msgspec

Siyet · 2026-04-09T19:16:26Z

Reopening #998 (auto-closed when fork was deleted).

Fixes #683.

Previously, the numeric constraints gt, ge, lt, le, and multiple_of in msgspec.Meta were only valid for int and float types. This PR extends support to decimal.Decimal.

PR #692 by @cocorigon has been open since May 2024. The author has since explicitly declined to continue it and invited others to take it over. This is a fresh implementation with documentation and full test coverage.

Changes

_core.c: Added five new TypeNode constraint flags backed by PyObject* slots storing Decimal instances. A new ms_check_decimal_constraints() function is called after every Decimal decode path.
__init__.pyi: Updated Meta.__init__ parameter types to include Decimal.
docs/constraints.rst: Updated Numeric Constraints section to mention decimal.Decimal.
tests/unit/test_constraints.py: Added TestDecimalConstraints covering all constraint types.

Notable design decisions

multiple_of for Decimal uses Python's % operator, which gives exact arithmetic.
Constraint bounds passed as float are converted to Decimal via string representation to preserve precision.

ofek · 2026-04-09T21:32:40Z

It looks like the Git history needs to be fixed on this one.

Siyet · 2026-04-10T12:08:25Z

The failing build job here is unrelated to this PR: it's the link checker tripping on the Pydantic docs redirect (docs.pydantic.dev/latest/ → pydantic.dev/docs/validation/...), which is fixed in #1008. Once #1008 lands and this branch is rebased, CI should go green.

Siyet · 2026-04-10T12:28:35Z

@ofek done — force-pushed a clean rewrite of the branch:

Squashed into a single commit with a proper message and Closes #683 reference.
Rebased onto current main, dropping all the leftover commits from the deleted fork (CODEOWNERS, package rename, doc-link rewrites, plus changes that belong to other PRs).
Scope is now strictly the four Decimal-relevant files: _core.c, __init__.pyi, docs/constraints.rst, tests/unit/test_constraints.py. Diff stat is +353/-52.
MS_CONSTR_DECIMAL_* flags claim bits 36-40, the lowest unused slots in TypeNode->types. If Support Literal[True] and Literal[False] types #1004 lands first, I'll rebase to whatever bits are next free.

Build job is still red on the link checker but that's the unrelated #1008 fix; nothing in this branch touches it.

Extends gt, ge, lt, le, and multiple_of Meta constraints to work with decimal.Decimal types, enabling exact arithmetic comparisons and sub-integer precision constraints without floating point issues. Constraint bounds passed as int or float are coerced to Decimal via their string representation to preserve precision. multiple_of uses Python's % operator for exact arithmetic. Closes #683

Siyet · 2026-04-20T19:22:46Z

Rebased onto main now that #1004 has landed. #1004 took bits 36-37 for MS_TYPE_BOOLLITERAL_TRUE/FALSE, so I shifted the Decimal constraint flags to the next free slots:

MS_CONSTR_DECIMAL_GT/GE/LT/LE -> bits 38-41 (the remaining gap between types and existing constraints)
MS_CONSTR_DECIMAL_MULTIPLE_OF -> bit 61 (the only free slot that's left below the reserved MS_EXTRA_FLAG at 63)

No other constraint bits were moved. SLOT numbering and the TypeNode->details[] ordering are unchanged (SLOTs use the named constants, not raw bit positions).

provinzkraut

I think this should have a check that disallows Decimal for gt, ge, etc. constraints where the type is not also a Decimal

Coercing a Decimal bound to int or float silently drops precision, which defeats the point of specifying the bound as a Decimal in the first place. Only allow Decimal bounds when the annotated type is also Decimal, and raise a TypeError otherwise at TypeNode construction time.

Siyet · 2026-04-21T07:25:47Z

@provinzkraut added the check: a Decimal in gt/ge/lt/le/multiple_of now raises TypeError at encoder construction time when the annotated type isn't Decimal. Coercing to int/float loses precision, so a Decimal bound there is pointless.

Covered with a parametrized test, docs updated.

sobolevn · 2026-06-15T10:05:13Z

-        lt: Union[int, float, None] = None,
-        le: Union[int, float, None] = None,
-        multiple_of: Union[int, float, None] = None,
+        gt: Union[int, float, Decimal, None] = None,


This can now be a type alias, since it is repeated quite a lot and can be changed.

Done, pulled it out into _NumericBound = int | float | Decimal | None.

sobolevn · 2026-06-15T10:07:44Z

            return false;
        }
    }
+    else if (PyObject_IsInstance(val, mod->DecimalType)) {


This allows Decimal subclasses, but you never test them. Since int and float subclasses are not allowed above - what is the correct way to handle this?

int/float use *_CheckExact because bool is an int subclass (we don't want to silently accept True as a numeric bound). Decimal has no such footgun subclass, and _constr_as_decimal normalizes the bound through Decimal(obj) at TypeNode-build time anyway, so a subclass's overridden dunders never reach the checks (verified with a hostile subclass overriding __gt__/__mod__ to raise: they're never called). Accepting subclasses is intentional; added test_decimal_subclass_bound.

sobolevn · 2026-06-15T10:40:13Z

        if ((out = TypeNode_traverse(node, visit, arg)) != 0) return out;
    }
+    /* Traverse Decimal constraint PyObject* */
+    if (self->types & (MS_CONSTR_DECIMAL_GT | MS_CONSTR_DECIMAL_GE)) {


This should be done before return out. Otherwise it can be left in a broken state.

Agreed, moved the Decimal-constraint Py_VISITs before the loops that can return out early. They're direct references on this node, so visiting them before recursing into children is the right place. (The Decimal pointers aren't counted in n_obj, so there's no double-visit.)

sobolevn · 2026-06-15T10:45:28Z

+        PyObject *c = TypeNode_get_constr_decimal_min(type);
+        int ok = PyObject_RichCompareBool(obj, c, Py_GT);
+        if (ok == -1) return false;
+        if (MS_UNLIKELY(!ok)) {


I don't think that it is unlikely. I would say that ok == -1 is unlikely.

The existing ms_decode_constr_int (and the float/str checks) mark the validation-failure path !ok as MS_UNLIKELY since valid data is the hot path, so I kept that for consistency. You're right about the error path though: marked ok < 0 as MS_UNLIKELY (and switched == -1 to < 0) in all four comparison blocks.

sobolevn · 2026-06-15T10:46:05Z

+        PyObject *modulo = PyNumber_Remainder(obj, c);
+        if (modulo == NULL) return false;
+        PyObject *zero = PyLong_FromLong(0);
+        if (zero == NULL) { Py_DECREF(modulo); return false; }


Suggested change

if (zero == NULL) { Py_DECREF(modulo); return false; }

if (zero == NULL) {

Py_DECREF(modulo);

return false;

}

style nit.

Dropped this block entirely, there's no zero to allocate anymore (see the multiple_of comment below).

sobolevn · 2026-06-15T10:48:13Z

+        PyObject *c = TypeNode_get_constr_decimal_multiple_of(type);
+        PyObject *modulo = PyNumber_Remainder(obj, c);
+        if (modulo == NULL) return false;
+        PyObject *zero = PyLong_FromLong(0);


You don't need to do the second compare in Python, you can do this in C.

Convert modulo to PyNumber_AsSsize_t (or whatever) and compare it with == 0 :)

PyNumber_AsSsize_t won't work here: Decimal has no __index__, so it'd raise TypeError on any remainder (including integral ones). Switched to PyObject_IsTrue(modulo) instead, a Decimal is falsy iff it's zero (including -0, 0E+10). That drops the zero allocation and stays correct for fractional remainders like Decimal("0.5").

sobolevn · 2026-06-15T10:50:50Z

+
+    `decimal.Decimal` bounds (``gt``, ``ge``, ``lt``, ``le``, ``multiple_of``)
+    are only valid on `decimal.Decimal` types. Applying a `decimal.Decimal`
+    bound to an ``int`` or ``float`` type raises a `TypeError`, since coercing


Sorry, I have a question about int here. How does it lose precision?

Good catch, "lose precision" was imprecise for int. It's not truncation: the bound would be converted via PyLong_AsLongLongAndOverflow, which needs __index__, and Decimal has none, so any Decimal (Decimal("2") as well as Decimal("1.5")) is rejected by the int conversion, and a fractional one has no integer equivalent at all. For float the bound would be rounded to the nearest binary float. Reworded the note to drop the incorrect "truncate".

# Conflicts: # src/msgspec/__init__.pyi # tests/unit/test_constraints.py

Siyet · 2026-06-15T23:31:26Z

@provinzkraut this is already handled in this PR: a Decimal bound on an int/float type is rejected with a TypeError (typenode_collect_constraints, via PyObject_IsInstance(..., DecimalType)), covered by test_invalid_decimal_bound_on_non_decimal for all five of gt/ge/lt/le/multiple_of. The check runs at TypeNode construction, so it's protocol-agnostic (msgpack behaves identically).

Siyet requested review from jcrist and ofek April 10, 2026 07:32

Siyet mentioned this pull request Apr 10, 2026

Support numeric constraints for Decimal types #692

Closed

Siyet force-pushed the decimal-constraints-683 branch from 8c80626 to f406983 Compare April 10, 2026 12:28

Siyet force-pushed the decimal-constraints-683 branch from d2ce46f to 023b9e1 Compare April 20, 2026 19:22

provinzkraut reviewed Apr 20, 2026

View reviewed changes

sobolevn reviewed Jun 15, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into decimal-constraints-683

b449815

# Conflicts: # src/msgspec/__init__.pyi # tests/unit/test_constraints.py

Siyet temporarily deployed to docs-preview June 15, 2026 22:43 — with GitHub Actions Inactive

Address review feedback on Decimal constraints

72b7932

Siyet temporarily deployed to docs-preview June 15, 2026 23:31 — with GitHub Actions Inactive

Format test_decimal_subclass_bound

f59ed16

Siyet deployed to docs-preview June 15, 2026 23:39 — with GitHub Actions View deployment

Conversation

Siyet commented Apr 9, 2026

Changes

Notable design decisions

Uh oh!

ofek commented Apr 9, 2026

Uh oh!

Siyet commented Apr 10, 2026

Uh oh!

Siyet commented Apr 10, 2026

Uh oh!

Siyet commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

provinzkraut left a comment

Choose a reason for hiding this comment

Uh oh!

Siyet commented Apr 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Siyet commented Jun 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Siyet commented Apr 20, 2026 •

edited

Loading