Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
997ce44
Reject record UDT pairs whose leaf counts differ
eriknw Jul 31, 2026
480f08f
Ignore the UDT repr size warning in the nesting mismatch test
eriknw Aug 4, 2026
1a7db6c
Compare record leaf paths and field shapes, not leaf counts
eriknw Oct 4, 2026
4210418
Type UDT ops like built-in ops, cast UDT stores, broadcast array UDTs
eriknw Oct 7, 2026
3bb5602
Cast UDT stores without temporaries, type UDT literals, fold in #592-…
eriknw Oct 8, 2026
66effa1
Type Python numbers beside built-in dtypes as numpy 2 does (NEP 50)
eriknw Oct 8, 2026
8e20881
Compare select value thunks exactly; isequal compares arrays as np.ar…
eriknw Oct 8, 2026
6690814
Compare a Python int beyond a 64-bit element's range exactly
eriknw Oct 8, 2026
3ffd904
Give each UDT layout its own JIT C name; Scalar == without Numba; CI …
eriknw Oct 9, 2026
b6fdcec
Spell out complex * and / for UDTs; Numba 0.68 follows Python 3.14
eriknw Oct 9, 2026
428d00f
Move UDT and UDF logic out of the general modules
eriknw Oct 14, 2026
54d3b77
Pass CODECOV_TOKEN to the coverage upload
eriknw Oct 20, 2026
67a7033
Cover how core/literals.py types literal operands
eriknw Oct 20, 2026
70d9bac
Cover select's rewrite of value thunks of another type
eriknw Oct 20, 2026
9513655
Cover UDT store casts, ewise_add casts and fused stores
eriknw Oct 20, 2026
9746928
Cover UDF return-type errors and the Numba error summary
eriknw Oct 20, 2026
1abf250
Cover the JIT layout checks for records a C struct cannot match
eriknw Oct 20, 2026
8e977df
Remove branches no caller can reach in the UDT code
eriknw Oct 20, 2026
95999ec
Exclude Numba-compiled bodies and unreachable guards from coverage
eriknw Oct 20, 2026
3f8ed9f
Drop test conditions that cannot change and a needless Numba skip
eriknw Oct 20, 2026
8890be9
Fix select on complex values with a thunk of another type
eriknw Oct 20, 2026
dd44ae8
Compare UDTs that are neither records nor arrays in eq and ne
eriknw Oct 20, 2026
bee1f0f
Type each product of Matrix.power for its UDT operands
eriknw Oct 20, 2026
e64fd32
Select with a float64 thunk too large for FP32 without a warning
eriknw Oct 20, 2026
75f73b8
Refuse values that a bytes, date or duration leaf of a UDT would not …
eriknw Oct 20, 2026
4fa58e6
Type every leaf of a record literal, numpy numbers, text and dates too
eriknw Oct 20, 2026
31eccfd
Refuse UDT ops on float16 elements by name; compare them in numpy for…
eriknw Oct 20, 2026
6ab263e
Type a record's array field as an array UDT types its literal
eriknw Oct 20, 2026
fbf7ba7
Refuse == between text, dates and numbers with eq's KeyError
eriknw Oct 20, 2026
f8b1002
Test the text-date and duration-for-a-date branches of record literal…
eriknw Oct 20, 2026
44abf82
Let a UDF return an operand of a UDT that is neither a record nor an …
eriknw Oct 20, 2026
fc670c9
Keep text that a bare text UDT would cut short out of it, and compare…
eriknw Oct 20, 2026
9536335
Keep a NaT duration NaT in another unit on numpy before 2.3
eriknw Oct 20, 2026
ff9c433
Keep the JIT layout test's record short enough to name on SuiteSparse 8
eriknw Oct 21, 2026
f6d9129
Convert a NaT duration in its leaf's unit, not as a NaT of no unit
eriknw Oct 21, 2026
16f679c
Skip the JIT part of the one-name UDT layouts test without a C compiler
eriknw Oct 27, 2026
2a564c2
Expand an iso array UDT's value without an extra axis in exports
eriknw Oct 28, 2026
8a109a5
Refuse a zero-size UDT, so a str literal raises TypeError on SuiteSpa…
eriknw Oct 28, 2026
7efd40b
Note that SuiteSparse:GraphBLAS 10.5.2 fixed select with a value op
eriknw Oct 28, 2026
1034473
Say which array Numba cannot type when registering a UDT
eriknw Oct 29, 2026
0a6c4b5
Type an IndexBinaryOp bound to theta when it runs, with theta of its …
eriknw Oct 29, 2026
102f3a0
Refuse a record literal whose array field part does not broadcast to …
eriknw Oct 29, 2026
530932a
Cover more literals a UDT refuses and the Numba error of a lifted op
eriknw Oct 29, 2026
83643e1
Give each type signature of a JIT IndexBinaryOp its own C name
eriknw Nov 3, 2026
cd6e0a8
Name each op registered from C by its full name and definition
eriknw Nov 3, 2026
ea63892
Refuse an empty Scalar theta for an IndexBinaryOp
eriknw Nov 3, 2026
8cb81c3
Leave a dtypes.ss.register_new name free when the registration fails
eriknw Nov 3, 2026
b331963
Type a Python date or duration in a record literal as numpy does
eriknw Nov 3, 2026
cf64e89
Compare a UDT of dates or durations with one in another unit as numpy…
eriknw Nov 3, 2026
2385b58
Type a literal beside a typed op in the op's own input type
eriknw Nov 3, 2026
7afa685
Accumulate into a UDT Scalar as into a built-in Scalar
eriknw Nov 3, 2026
658b522
Pair an array literal with an array UDT when their shapes broadcast
eriknw Nov 3, 2026
339fd93
Give IndexBinaryOp is_positional, so dtype in op and op[dtype] work
eriknw Nov 3, 2026
82f3312
Refuse a bound IndexBinaryOp in reduce with the usual TypeError
eriknw Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/test_and_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,11 @@ jobs:
coverage report --show-missing
- name: codecov
uses: codecov/codecov-action@v5
with:
# Codecov takes tokenless uploads only from forks. A PR from a branch of this
# repository, or a push to main, is refused with "Token required because
# branch is protected".
token: ${{ secrets.CODECOV_TOKEN }}
- name: Notebooks Execution check
if: matrix.slowtask == 'notebooks'
run: |
Expand Down
1 change: 0 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ repos:
- id: check-json # no JSON files yet; enabled for the future
- id: check-toml
- id: check-yaml
- id: check-executables-have-shebangs
- id: check-vcs-permalinks
- id: debug-statements
- id: destroyed-symlinks
Expand Down
8 changes: 4 additions & 4 deletions docs/user_guide/operators.rst
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,8 @@ receives both values, the indices of *each* value, and a thunk parameter:
There are no built-in IndexBinary operators; all are user-defined (see
:doc:`udf`). Binding a ``theta`` value to an IndexBinaryOp produces a
regular BinaryOp usable directly in ``ewise_mult`` / ``ewise_add``, or as
the multiplier of a Semiring for ``mxm`` / ``mxv`` / ``vxm``.
the multiplier of a Semiring for ``mxm`` / ``mxv`` / ``vxm``. Like other
BinaryOps, its type is chosen when it runs, from both inputs and ``theta``.

Example usage:

Expand All @@ -243,11 +244,10 @@ Example usage:

gb.indexbinary.register_new("discounted_sum", discounted_sum)

binop = gb.indexbinary.discounted_sum[float](0.5) # bind theta
binop = gb.indexbinary.discounted_sum(0.5) # bind theta
C << A.ewise_mult(B, binop)

# For mxm, wrap in a Semiring; the additive monoid must accept the
# bound op's return type.
# For mxm, wrap in a Semiring
sr = gb.semiring.register_anonymous(gb.monoid.plus, binop)
D << A.mxm(B, sr)

Expand Down
27 changes: 27 additions & 0 deletions docs/user_guide/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,33 @@ When a data type is needed in an API call, a string or numpy or numba dtype may
instead of the actual data type object. Additionally, the Python builtin
``bool``, ``int``, and ``float`` may be used. ``int`` indicates INT64 and ``float`` indicates FP64.

Python numbers in operations
----------------------------

A Python ``bool``, ``int``, ``float`` or ``complex`` used as an operand, such as
the ``1`` in ``v + 1`` or ``v.apply(binary.times, right=0.5)``, is *weak*, as in
numpy 2 (NEP 50). It takes the other operand's data type when that type's kind
can hold it, and its own default type (BOOL, INT64, FP64 or FC64) when not. So
with an INT8 vector ``v``, ``v + 1`` is INT8 and ``v * 0.5`` is FP64, while
``v + 300`` raises ``OverflowError`` rather than wrapping, and ``v * 2`` wraps
in INT8 as it does in numpy. An FP32 vector times ``0.5`` is FP32, plus ``1j``
is FC32, and plus ``1e300`` is infinity, with a ``RuntimeWarning`` as in numpy. numpy scalars, 0-d arrays and ``Scalar`` objects keep their own
type: ``v + np.int64(1)`` is INT64. python-graphblas applies this rule itself,
so it does not depend on the installed version of numpy. User-defined types
follow the same rule, field by field (see :doc:`udt`).

A comparison has no result type to keep, so an integer outside the operand's
range compares exactly, as in numpy: ``v < 300`` is True for every element, as
is ``w < 2**63`` for an INT64 vector ``w``.
Nor does ``v / 300`` raise, since its result is FP64 whatever the integer. An
FP32 vector ``== 0.1`` compares in FP32, and ``== 1e300`` compares with
infinity, with a ``RuntimeWarning`` as in numpy. The defaults of ``ewise_union``
follow the rule, and so does the ``fill_value`` of ``to_dense``, except that a
value that does not fit widens the result instead of raising. A literal given
to a typed operator, such as ``binary.plus["INT64"]``, takes that operator's
type, and a ``select`` or ``IndexUnaryOp`` thunk that is not compared with the
values, such as a row index, keeps its own type.

User-defined Types
------------------

Expand Down
52 changes: 42 additions & 10 deletions docs/user_guide/udf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,11 +125,43 @@ works in ``ewise_mult``, ``ewise_add``, and the other elementwise paths:

indexbinary.register_new("discounted_sum", discounted_sum)

bound = indexbinary.discounted_sum[float](0.5) # theta = 0.5
A = Matrix.from_coo([0, 1], [0, 1], [1.0, 2.0])
B = Matrix.from_coo([0, 1], [0, 1], [3.0, 4.0])
bound = indexbinary.discounted_sum(0.5) # theta = 0.5
A = Matrix.from_coo([0, 1], [0, 1], [1, 2])
B = Matrix.from_coo([0, 1], [0, 1], [3, 4])
C = A.ewise_mult(B, bound).new()
# C[0,0] = (1+3)*0.5 = 2; C[1,1] = (2+4)*0.5 = 3
# C is FP64: C[0,0] = (1+3)*0.5 = 2.0; C[1,1] = (2+4)*0.5 = 3.0

Like any other BinaryOp, the bound op takes its type each time it runs: the
type that holds both inputs and ``theta``, as ``apply`` types an IndexUnaryOp
from its input and thunk. One type serves x, y and ``theta``. ``theta`` has
the type of its value: a Python int is INT64 and a float FP64, and a numpy
scalar or a ``Scalar`` keeps its own type. With no ``theta``, ``theta`` is 0
(``False``), which changes no type. Above, the INT64 matrices and the float
``theta`` run in FP64. To choose the type yourself, name it:
``indexbinary.discounted_sum[float](0.5)``, or
``indexbinary.discounted_sum(0.5, dtype=float)``. The inputs and ``theta`` are
then cast to that type.

An op registered with ``is_udt=True`` is compiled for the types of x, y and
``theta`` as they are, so none of them is cast: ``op(0.5)`` on INT64 inputs
gets the float 0.5. A numpy array ``theta`` has the array UDT of its dtype
and shape. A tuple has no type of its own, so give a record or array
``theta`` as a ``Scalar`` of its UDT (``Scalar.from_value((1, 2.5), udt)``).
``op[X, Y, T](theta)`` names all three types, and ``op[X, Y](theta)`` gives
``theta`` the type of y.

This differs from an IndexUnaryOp in ``apply``, which makes a number thunk an
element of the UDT data's type. Beside UDT data, a number ``theta`` stays a
number: with ``x + theta`` on INT32[3] data, ``op(5)`` adds an INT64 to each
element, which gives an int64 array, not an INT32[3] element, so the op
raises ``UdfParseError`` when it is used. To make ``theta`` the data's type,
pass a ``Scalar`` of that type, or a numpy value of the element type
(``op(np.int32(5))``), or name the type: ``op(5, dtype=udt)`` binds
``op[udt](5)``, which makes 5 an element of ``udt``.

A ``Scalar`` ``theta`` is copied when it is bound, so changing the ``Scalar``
afterwards does not change the bound op. An ``apply`` expression, by
contrast, reads its ``Scalar`` thunk when it runs.

To use a bound IndexBinaryOp as the multiplier in ``mxm`` / ``mxv`` /
``vxm``, wrap it in a Semiring. ``Semiring.register_anonymous`` accepts a
Expand All @@ -142,12 +174,12 @@ bound IBO directly:
sr = semiring.register_anonymous(monoid.plus, bound)
C = A.mxm(B, sr).new()

The resulting Semiring is monomorphic in the bound IBO's input/output types
(SuiteSparse builds exactly one ``GrB_Semiring`` for that type pair, rather
than the type-polymorphic table the standard semirings carry). To reuse the
same IBO at a different type, bind theta again under that type and build a
new Semiring. Per SuiteSparse, monoids themselves cannot be built from an
IndexBinaryOp; only the multiplier slot accepts one.
The Semiring is typed each time it runs, as the bound op is. One built from
an op bound at a type (``discounted_sum[float](0.5)``) is monomorphic in that
op's input/output types instead (SuiteSparse builds exactly one
``GrB_Semiring`` for that type pair, rather than the type-polymorphic table
the standard semirings carry). Per SuiteSparse, monoids themselves cannot be
built from an IndexBinaryOp; only the multiplier slot accepts one.

IndexBinaryOps require SuiteSparse:GraphBLAS >= 9.4.

Expand Down
155 changes: 149 additions & 6 deletions docs/user_guide/udt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ Field type rules:
struct) are supported as fields, and built-in operators lift through them.
In the dict and dataclass forms, write an array field as ``"FP64[3]"``; the
dict form also takes an existing UDT, such as ``{"id": int, "pos": point3}``.
- Numba cannot type an array of ``bool``, ``datetime64`` or ``timedelta64``, so
with Numba installed, registering a UDT that holds one raises ``ValueError``.
Use an array of ``uint8`` for ``bool``, and of ``int64`` for the others.
- Strings and Python objects are not supported.
- Dataclass annotation strings (e.g., ``"int"``) resolve through
``lookup_dtype``.
Expand Down Expand Up @@ -134,12 +137,152 @@ For example:
plus_times = semiring.plus_times[edge_dtype] # field-wise multiply-add semiring
total = v.reduce(agg.sum[edge_dtype]).new() # field-wise reduce

The lift is field-by-field for record UDTs and element-by-element for array
UDTs. Field types that don't support the operation raise ``KeyError`` on the
first lookup. ``binary.min``, ``binary.max``, and ``binary.floordiv`` reject
UDTs with any complex leaf (no ordering, no integer modulus); use ``plus``,
``minus``, ``times``, or ``truediv`` for complex arithmetic, or register a
custom binary op.
How results are typed
~~~~~~~~~~~~~~~~~~~~~

A lifted op types each field or element as it types a Vector of that built-in
dtype, so it never narrows. Below, ``v`` is a Vector of an ``INT8[3]`` UDT.

.. list-table::
:header-rows: 1
:widths: 25 45 30

* - Operands
- Rule
- Example
* - One UDT
- Each field or element as on its built-in dtype.
- ``v / v`` is ``FP64[3]``
* - Two UDTs
- Records pair when they have the same field names, in the same order, at
every level, and the same shape for each field. Array UDTs pair when
their shapes broadcast as numpy's do. Any other pair raises
``KeyError``.
- ``INT8[3]`` with ``INT64[2, 3]`` gives ``INT64[2, 3]``
* - Result type
- An operand's type when it holds the result; else the promoted layout,
which is the UDT registered with that layout if there is one, else an
anonymous UDT.
- ``{x: INT64, y: FP64}`` with ``{x: FP64, y: INT64}`` gives
``{x: FP64, y: FP64}``
* - A built-in Vector, ``Scalar``, numpy scalar or numpy array
- Strong: keeps its dtype. A scalar combines with every field or element;
an array pairs as an array UDT of its own shape. A record's array field
keeps its shape, so a literal for it must broadcast to that shape, or
``ValueError`` names both: ``np.array([[1, 2, 3]])`` does not fit an
``INT8[3]`` field, though beside an ``INT8[3]`` UDT it gives
``INT64[1, 3]``.
- ``v + np.int64(1)`` is ``INT64[3]``
* - A Python number, or a tuple, list or dict of them laid out like an
element
- Weak, field by field: a field's dtype when its kind holds the number,
else INT64, FP64 or FC64 (numpy 2's NEP 50 rule, on numpy 1 too, and
extended to sequences). An int out of an integer field's range raises
``OverflowError``, except for ``truediv``, whose fields are floats
whatever the int (``v / 300`` is ``FP64[3]``).
- ``v + 1`` is ``INT8[3]``, ``v * 2.5`` and ``v + (0.5, 1, 2)`` are
``FP64[3]``, ``v + 300`` raises
* - ``eq`` and ``ne``
- Type a literal as above, as built-in comparisons do, except that a
literal with an int out of a field's range equals no element, rather
than raise. Return ``BOOL``. Array elements broadcast, as numpy's ``==``
does, and so does ``==`` on a Scalar, which returns a bool. ``isequal``
compares as ``np.array_equal``: elements must have the same shape,
apart from leading axes of length 1.
- ``v == 300`` is ``False``; an ``FP32[3]`` UDT ``== 0.1`` compares in
FP32; a Scalar ``s`` of ``[1, 1, 1]`` has ``s == 1`` but not
``s.isequal(1)``
* - A literal that must become an element: beside a user-defined op, an
``IndexUnaryOp`` thunk, an ``ewise_union`` default
- Its type by the rules above must fit the UDT: the UDT itself, or a
numpy value or ``Scalar`` that casts to it safely. A weak int does not
fit a bool field: write ``True`` or ``False``. A value that does not fit
but converts exactly, such as ``0`` for a bool field or ``2.0`` for an
int field, still converts with a ``DeprecationWarning`` and will raise
in a future version; any other raises ``ValueError``, and a float too
large for a float field raises ``OverflowError``.
- ``v.ewise_union(w, binary.plus, 0.5, 0)`` raises

A Monoid used element-wise (``apply``, ``ewise_mult``, ``ewise_union``) uses
its BinaryOp, for literals and for two different UDTs. ``min``, ``max`` and ``floordiv`` reject UDTs
with a complex field; use a custom op for those.

Each element, including each element of an array field, is computed as the
built-in op computes it, and the Numba cfunc and a C JIT kernel give the same
bits: ``min`` and ``max`` ignore NaN (C's ``fmin`` and ``fmax``) and resolve a
tie between ``-0.0`` and ``0.0`` to ``-0.0`` for ``min`` and ``0.0`` for
``max``, ``floordiv`` is numpy's, with 0 for an integer divided by zero, and
``truediv`` is numpy's, with numpy's infinities for a zero divisor. A complex
quotient takes Smith's method, as numpy's does, but divides where numpy
multiplies by a reciprocal, so it can differ from numpy's in the last bit (and
stays finite where numpy's overflows). A few edge cases can differ from built-in
vectors, where the UDT gives numpy's answer: ``INT64_MIN // -1`` is
``INT64_MIN`` (an ``INT64`` vector gives 0), a complex product or quotient with
an infinite or NaN part, a complex quotient with a ``-0.0`` divisor, and the
signed-zero tie, which SuiteSparse's built-in ``min`` leaves to the C library.

Storing results
~~~~~~~~~~~~~~~

GraphBLAS casts built-in dtypes when it stores a result, but it cannot cast a
UDT, so python-graphblas does it, without ever making a temporary copy of the
data.

.. list-table::
:header-rows: 1
:widths: 40 60

* - Store
- What happens
* - Into an object of another UDT type of the same layout: records that
pair as operands do, or arrays of the same shape apart from leading
axes of length 1
- Each field or element is cast as GraphBLAS casts that pair of built-in
dtypes: integers wrap; a float stored as an integer truncates toward
zero, saturates at the bounds, and is 0 if NaN; a complex number loses
its imaginary part; anything nonzero is ``True``.
* - ...from an existing object: ``w << v``, ``v.dup(dtype=...)``, with a
mask, an accumulator or ``replace``
- One pass through a cast op.
* - ...from an element-wise lifted op: ``apply``, ``ewise_mult``,
``ewise_union``, and infix operators that use them
- The op writes the object's type itself, in the same call:
``int_udt_vec += 0.5`` truncates, as it does for an ``INT8`` vector.
* - ...from anything else: ``mxm``, ``reduce`` into a Vector, ``ewise_add``,
a user-defined op, or a whole Vector or Matrix assigned into part of an
object
- ``DomainMismatch``: this would need a converted copy of the whole
result. Make it explicitly: compute the result, then store it
(``w << expr.new()``); to assign, convert the value with
``.dup(dtype=...)`` first.
* - ...from a ``Scalar`` or one element
- Computed in its own type, then cast. With an accumulator, the value is
cast before it accumulates, as GraphBLAS converts a value of a built-in
dtype, and so is a literal: an ``INT8[3]`` Scalar's ``s += 0.5`` adds 0,
as an ``INT8`` Scalar's does.
* - Between a UDT and a built-in dtype, or layouts that do not correspond
- ``DomainMismatch``.

For the same reason, ``ewise_add`` on Vectors and Matrices raises
``DomainMismatch`` when an operand's type differs from the result's, as for two
different UDTs or ``truediv`` on an integer UDT: it would copy a value that has
no partner into the result type. ``ewise_union`` and ``ewise_mult`` work. Infix
``+`` on two such operands takes the union instead, with each operand's zero for
a missing value, as ``-`` takes 0. That zero is ``-0.0`` in a float field, so an
entry of one operand alone keeps its value, as ``ewise_add`` would copy it;
only a ``-0.0`` beside an integer field of the other operand becomes ``0.0``. A
Scalar is one element, so ``ewise_add`` converts its operands to the result type
instead.

.. code-block:: python

v = Vector(int8x3, size=2) # an INT8[3] UDT
f = Vector(fp32x3, size=2) # an FP32[3] UDT
(v * 2.5).new() # FP64[3]: the literal is weak, 2.5 needs FP64
(v * f).new() # FP32[3], as INT8 times FP32
v += 1 # stays INT8[3]
v << v * 2.5 # computes in FP64, stores as INT8 (truncates)
(v + f).new() # FP32[3], by ewise_union; v.ewise_add(f) raises

Composite aggregators (``agg.hypot``, ``agg.L1norm``, ``agg.Linfnorm``,
``agg.sum_of_squares``, ``agg.sum_of_inverses``) do *not* auto-lift to UDTs;
Expand Down
Loading
Loading