Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
dbbe0c8
Cover date-field text, a Vector accumulated into a UDT Scalar and more
eriknw Nov 3, 2026
7e573f6
Index a Monoid with two UDTs as with two built-in types
eriknw Nov 3, 2026
d8830d6
Take an out-of-range int exactly beside any op whose result has anoth…
eriknw Nov 3, 2026
e484f7f
Say in udt.rst what a UDT of text or dates supports
eriknw Nov 3, 2026
7c1b7ae
Say how lifted ops compute a record's array field, in _iter_record_le…
eriknw Nov 3, 2026
3193e0a
Name Scalar.__eq__ in the error for a value that is not a literal
eriknw Nov 3, 2026
44537e2
Call a stored or accumulated object a value in the store-cast error
eriknw Nov 3, 2026
a82cc5f
Index monoid.any with a UDT pair by the type the pair promotes to
eriknw Nov 3, 2026
0a95cef
Compare with infinity only for a built-in comparison, not an op named…
eriknw Nov 3, 2026
2aad6df
Raise for a user's op that would meet an int beyond 64 bits in a floa…
eriknw Nov 3, 2026
3648473
Read a dict as a record only beside a record UDT in ==
eriknw Nov 3, 2026
0d80691
Say that operators on a text or date UDT need Numba
eriknw Nov 3, 2026
4572ee5
Compare indices with the bound given in select.row, column and index
eriknw Nov 3, 2026
9595bf1
Count every op python-graphblas defines at init as standard, truediv …
eriknw Nov 3, 2026
cb6fbf4
Take the thunk's value as an index bound that is not a number
eriknw Nov 3, 2026
e4d3717
Give a select helper's left-operand comparison the op's TypeError again
eriknw Nov 3, 2026
4062382
Select no index for a NaN bound in the row, column and index helpers
eriknw Nov 3, 2026
65fc07c
Give the NaT in test_udt_literal_converts_only_when_exact a unit
eriknw Nov 4, 2026
4017c62
Ignore _test_weak_lt in test_operator_types like other test-made ops
eriknw Nov 4, 2026
47c00c9
Give text read as NaT the date field's unit
eriknw Nov 4, 2026
a2a6f57
Type a Monoid indexed with a UDT pair without compiling plus
eriknw Nov 10, 2026
2f14c97
Keep the result message for a computed result assigned to part of an …
eriknw Nov 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/user_guide/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,14 @@ follow the same rule, field by field (see :doc:`udt`).
A comparison has no result type to keep, so an integer outside the operand's
range compares exactly, as in numpy: ``v < 300`` is True for every element, as
is ``w < 2**63`` for an INT64 vector ``w``.
Nor does ``v / 300`` raise, since its result is FP64 whatever the integer. An
Nor does an operator whose result has another type raise: ``v / 300`` is FP64
whatever the integer, and ``v.apply(binary.isclose, right=300)`` is False for
every element. A user-defined operator whose result has another type gets the
integer exactly where the operand's type and the integer's share an integer
type (INT16 for ``300`` here), and raises ``OverflowError`` where they would
share only a floating-point type, which can round the integer: beside an INT64
vector, ``2**63`` would meet the elements in FP64, where ``2**63 - 1`` and
``2**63`` are equal. An
FP32 vector ``== 0.1`` compares in FP32, and ``== 1e300`` compares with
infinity, with a ``RuntimeWarning`` as in numpy. The defaults of ``ewise_union``
follow the rule, and so does the ``fill_value`` of ``to_dense``, except that a
Expand Down
17 changes: 13 additions & 4 deletions docs/user_guide/udt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,19 @@ Field type rules:
struct) are supported as fields, and built-in operators lift through them.
In the dict and dataclass forms, write an array field as ``"FP64[3]"``; the
dict form also takes an existing UDT, such as ``{"id": int, "pos": point3}``.
- Numba cannot type an array of ``bool``, ``datetime64`` or ``timedelta64``, so
with Numba installed, registering a UDT that holds one raises ``ValueError``.
Use an array of ``uint8`` for ``bool``, and of ``int64`` for the others.
- Strings and Python objects are not supported.
- Numba cannot type an array of ``bool``, ``datetime64``, ``timedelta64``,
``str`` or ``bytes``, so with Numba installed, registering a UDT that holds
one raises ``ValueError``. Use an array of ``uint8`` for ``bool`` and
``bytes`` (one per byte), of ``uint32`` for ``str`` (one per character), and
of ``int64`` for the others.
- Text (``str`` or ``bytes``, such as ``np.dtype("S5")``), ``datetime64`` and
``timedelta64`` may be fields, or a whole UDT that is neither record nor
array. Such a UDT can be stored and extracted. With Numba, it can also be
compared (``==``, ``!=`` and ``isequal``), and ``first``, ``second``, ``pair``
and ``any`` work on it, but arithmetic operators do not lift to it: ``plus``
and ``min`` raise ``KeyError``. Without Numba, only a ``Scalar``'s ``==``,
``!=`` and ``isequal``, and ``pair``, work on it.
- Python objects (``np.dtype(object)``) are not supported.
- Dataclass annotation strings (e.g., ``"int"``) resolve through
``lookup_dtype``.

Expand Down
10 changes: 8 additions & 2 deletions graphblas/core/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -365,7 +365,7 @@ def _update(self, expr, mask=None, accum=None, replace=False, input_mask=None, *
expr = expr._extract_delayed()
elif type(expr) is type(self):
# Simple assignment (w << v)
if (cast_op := _store_cast_op(expr.dtype, self.dtype)) is not None:
if (cast_op := _store_cast_op(expr.dtype, self.dtype, is_value=True)) is not None:
expr = expr.apply(cast_op)
self._update(expr, mask, accum, replace, input_mask, opts=opts)
return
Expand Down Expand Up @@ -407,7 +407,9 @@ def _update(self, expr, mask=None, accum=None, replace=False, input_mask=None, *
from .matrix import Matrix, MatrixExpression, TransposedMatrix

if type(expr) is TransposedMatrix and type(self) is Matrix:
if (cast_op := _store_cast_op(expr.dtype, self.dtype)) is not None:
if (
cast_op := _store_cast_op(expr.dtype, self.dtype, is_value=True)
) is not None:
# C << A.T of another UDT type: the cast reads A transposed.
expr = expr.apply(cast_op)
else:
Expand Down Expand Up @@ -570,6 +572,7 @@ class BaseExpression:
"expr_repr",
"dtype",
"_value",
"_right_literal",
"__weakref__",
)
output_type = None
Expand Down Expand Up @@ -605,6 +608,9 @@ def __init__(
self.dtype = op.return_type
else:
self.dtype = dtype
# The right operand as the caller gave it, a literal a comparison typed
# beside the values; the select helpers compare indices with it instead.
self._right_literal = None
self._value = None

def new(self, dtype=None, *, mask=None, name=None, **opts):
Expand Down
63 changes: 44 additions & 19 deletions graphblas/core/literals.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,11 @@

import numpy as np

from .. import _STANDARD_OPERATOR_NAMES
from ..dtypes import FP64, lookup_dtype
from . import utils
from .dtypes import _nats_in_leaf_units, _view_if_same_layout
from .operator import IndexUnaryOp, SelectOp, TypedOpBase, binary_from_string
from .operator import IndexUnaryOp, ParameterizedUdf, SelectOp, TypedOpBase, binary_from_string
from .operator.binary import BinaryOp
from .operator.monoid import Monoid
from .udt import (
Expand Down Expand Up @@ -97,7 +98,7 @@ def _literal_operand(dtype, value, op, *, left=False):
# by a comparison or by an op that gives another type (truediv).
if isinstance(op, str):
op = binary_from_string(op)
if op.name in _COMPARISONS:
if op.name in _COMPARISONS and not op._anonymous:
return (math.inf if value > 0 else -math.inf), FP64
return value, literal_dtype
if getattr(op, "_udt_types", True) is None:
Expand All @@ -114,7 +115,9 @@ def _literal_operand(dtype, value, op, *, left=False):
exact = is_comparison or op.name == "truediv"
if is_comparison and not isinstance(value, (np.generic, np.ndarray)):
np_type = dtype.np_type
parts = _dict_to_record(np_type, value) if isinstance(value, dict) else value
parts = value
if isinstance(value, dict) and np_type.names is not None:
parts = _dict_to_record(np_type, value)
# First: making an int beyond its leaf infinity reshapes the part.
_refuse_misfit_fields(dtype, np_type, parts)
if _int_beyond_leaf(np_type, parts) is not None:
Expand Down Expand Up @@ -179,13 +182,15 @@ def _weak_builtin_literal_dtype(dtype, value, op=None):
# keep, so an int out of range is typed exactly, as for a comparison:
# int8_vec / 300 is FP64, as numpy divides integers in float64.
np_type = _weak_literal_leaf([value], kind, dtype.np_type, exact=True)
if np_type == dtype.np_type:
rv = dtype
else:
try:
rv = lookup_dtype(np_type)
except ValueError: # A complex literal where the backend has no complex types
return None
if np.result_type(dtype.np_type, np_type).kind not in "iu" and not _is_builtin_op(op):
# The two then unify to a float type, which rounds an int beyond 64
# bits: in FP64, the INT64 2**63 - 1 and the int 2**63 are equal. A
# built-in op that gives another type computes in float anyway (a
# comparison is given infinity instead), but a user's op may compare.
raise
# Where the backend has no complex types (vanilla), lookup_dtype registers a
# complex literal's dtype as a UDT, so the literal keeps its own type.
rv = dtype if np_type == dtype.np_type else lookup_dtype(np_type)
if kind != "b" and rv.np_type in _SINGLE_PRECISION:
parts = (value.real, value.imag) if kind == "c" else (float(value),)
try:
Expand All @@ -204,15 +209,29 @@ def _weak_builtin_literal_dtype(dtype, value, op=None):
return rv


def _is_builtin_op(op):
"""Whether python-graphblas defines ``op``, or a built-in parameterized op made it."""
if op._anonymous:
info = getattr(getattr(op, "orig_func", None), "_parameterized_info", None)
return info is not None and _is_builtin_op(info[0])
module = op._op_class._modname if isinstance(op, ParameterizedUdf) else op._modname
return f"{module}.{op.name}" in _STANDARD_OPERATOR_NAMES


def _gives_another_type(op, dtype):
"""Whether built-in ``op`` on two operands of built-in ``dtype`` gives another type.
"""Whether ``op`` on two operands of built-in ``dtype`` gives another type.

``truediv`` on integers gives FP64, and the numpy ufuncs that compute in
float give a float type, as numpy computes them in a float loop.
``truediv`` on integers gives FP64, ``isclose`` gives BOOL, and the numpy
ufuncs that compute in float give a float type, as numpy computes them in
a float loop. A user's op counts by the type it gives too. A
parameterized op is typed with its default parameters, as
``get_typed_op`` types it.
"""
if op is None or getattr(op, "_anonymous", True) or not hasattr(op, "__getitem__"):
if op is None:
return False
try:
if isinstance(op, ParameterizedUdf):
op = op()
return op[dtype].return_type != dtype
except Exception: # An op that does not take this dtype reports it itself, later.
return False
Expand Down Expand Up @@ -257,7 +276,7 @@ def _literal_type(dtype, value, *, exact=False):
if _keeps_value(value, np_type):
return dtype
return lookup_dtype(np.dtype(f"{np_type.kind}{max(len(value), 1)}"))
if np_type.kind in "mM" and (date := _as_numpy_date(value, np_type.kind)) is not None:
if np_type.kind in "mM" and (date := _as_numpy_date(value, np_type)) is not None:
if date.dtype.kind == np_type.kind and _keeps_value(date, np_type):
return dtype
return lookup_dtype(date.dtype)
Expand Down Expand Up @@ -611,7 +630,7 @@ def leaf_dtype(leaf, part, *, in_record=False):
# takes the field's type, and converting it reports what is wrong.
if isinstance(part, (str, bytes)) and base.kind in "SU":
own = np.dtype(f"{base.kind}{max(len(part), 1)}")
elif (date := _as_numpy_date(part, base.kind)) is not None:
elif (date := _as_numpy_date(part, base)) is not None:
part = date
own = date.dtype
if own.kind != base.kind:
Expand Down Expand Up @@ -674,21 +693,27 @@ def record_leaves(np_type, part):
return dtype if literal_type == np_type else lookup_dtype(literal_type)


def _as_numpy_date(value, kind):
def _as_numpy_date(value, leaf):
"""Return ``value`` as numpy's ``datetime64`` or ``timedelta64``, or ``None`` if it is neither.

A numpy date or duration is itself; a Python ``datetime.date`` or
``datetime.datetime`` is a date in days or microseconds and a
``datetime.timedelta`` a duration in microseconds, as numpy makes them;
text is a date in the unit it is written to (``"2020-01-01T12:00"`` is in
minutes) when ``kind``, the leaf's, is ``"M"`` and numpy can read it.
minutes) when ``leaf``, the numpy dtype it meets, is a date and numpy can
read it. Text numpy reads as NaT (``"NaT"`` in any case, or empty) is NaT
in the leaf's unit: numpy would give it no unit, which numpy 2.5 deprecates.
"""
if isinstance(value, (np.datetime64, np.timedelta64)):
return value
try:
if isinstance(value, datetime.timedelta):
return np.timedelta64(value)
if isinstance(value, datetime.date) or (kind == "M" and isinstance(value, (str, bytes))):
if isinstance(value, datetime.date):
return np.datetime64(value)
if leaf.kind == "M" and isinstance(value, (str, bytes)):
if value.lower() in ("nat", "", b"nat", b""):
return np.datetime64("NaT", np.datetime_data(leaf)[0])
return np.datetime64(value)
except ValueError: # text that is not a date; converting it reports that
pass
Expand Down
6 changes: 5 additions & 1 deletion graphblas/core/matrix.py
Original file line number Diff line number Diff line change
Expand Up @@ -2426,6 +2426,7 @@ def apply(self, op, right=None, *, left=None):

"""
method_name = "apply"
right_literal = None
extra_message = (
"apply only accepts UnaryOp with no scalars or BinaryOp with `left` or `right` scalar "
"or IndexUnaryOp with `right` thunk."
Expand Down Expand Up @@ -2495,6 +2496,7 @@ def apply(self, op, right=None, *, left=None):
expr_repr = "{1.name}.apply({op}, left={0._expr_name})"
elif left is None:
if type(right) is not Scalar:
right_literal = right
right, dtype = _literal_operand(self.dtype, right, op)
try:
right = Scalar.from_value(right, dtype, is_cscalar=None, name="")
Expand Down Expand Up @@ -2538,7 +2540,7 @@ def apply(self, op, right=None, *, left=None):
expr_repr = "{0.name}.apply({op}, right={1._expr_name})"
else:
raise TypeError("Cannot provide both `left` and `right` to apply")
return MatrixExpression(
expr = MatrixExpression(
method_name,
cfunc_name,
args,
Expand All @@ -2549,6 +2551,8 @@ def apply(self, op, right=None, *, left=None):
at=self._is_transposed,
bt=self._is_transposed,
)
expr._right_literal = right_literal
return expr

def select(self, op, thunk=None):
"""Create a new Matrix by applying ``op`` to each element of the Matrix
Expand Down
24 changes: 20 additions & 4 deletions graphblas/core/operator/utils.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from types import BuiltinFunctionType, FunctionType, ModuleType

from ... import (
_STANDARD_OPERATOR_NAMES,
backend,
binary,
config,
Expand All @@ -14,6 +15,7 @@
)
from ...dtypes import UINT64, lookup_dtype, unify
from ..expr import InfixExprBase
from ..udt import _monoid_pair_type
from .base import (
_SS_OPERATORS,
OpBase,
Expand Down Expand Up @@ -49,6 +51,18 @@
# the tuple-form ``__reduce__`` that re-registers the function in the
# child.
IndexBinaryOp._initialize()
# Every op the namespaces hold now is python-graphblas's own, also ones made
# outside ``register_new`` (cdiv, truediv, rtruediv, the semirings on cdiv) and
# ones made on first use (binom). Record them all as standard: they pickle by
# name, and ``literals._is_builtin_op`` counts them built-in, with no list to
# keep in step.
for _module in [binary, unary, monoid, semiring, indexunary, select, indexbinary]:
_modname = _module.__name__.rsplit(".", 1)[-1]
_STANDARD_OPERATOR_NAMES.update(f"{_modname}.{key}" for key in _module._delayed)
for _val in list(vars(_module).values()):
if isinstance(_val, (OpBase, ParameterizedUdf)) and not _val._anonymous:
_STANDARD_OPERATOR_NAMES.add(f"{_modname}.{_val.name}")
del _module, _modname, _val
except Exception: # pragma: no cover (debug)
# Exceptions here can often get ignored by Python
import traceback
Expand Down Expand Up @@ -78,12 +92,14 @@ def get_typed_op(op, dtype, dtype2=None, *, is_left_scalar=False, is_right_scala
return op[UINT64]
if op._udt_types is None:
raise KeyError(f"{op.name} does not work with ({dtype}, {dtype2})")
if kind == "binary" and isinstance(op, Monoid):
if isinstance(op, Monoid):
# A Monoid takes one type, but used element-wise it is its
# BinaryOp, which takes the pair: ``v.ewise_mult(w, monoid.plus)``
# works where ``v * w`` does. Indexing (``monoid.plus[X, Y]``)
# still raises.
return get_typed_op(op.binaryop, dtype, dtype2)
# works where ``v * w`` does.
if kind == "binary":
return get_typed_op(op.binaryop, dtype, dtype2)
# Indexed with the pair, it takes the type the pair promotes to.
return op[_monoid_pair_type(op.binaryop.name, dtype, dtype2)]
return op._compile_udt(dtype, dtype2)
# Generic case: try to unify the two dtypes
try:
Expand Down
34 changes: 28 additions & 6 deletions graphblas/core/scalar.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,12 +158,12 @@ def __eq__(self, other):
Without Numba, which ``binary.eq`` needs for UDTs, a UDT's values are
compared in numpy, which broadcasts them the same way.
"""
return _scalar_eq(self, other)
return _scalar_eq(self, other, "__eq__")

__hash__ = None

def __ne__(self, other):
return not _scalar_eq(self, other)
return not _scalar_eq(self, other, "__ne__")

def __bool__(self):
"""Truthiness check.
Expand Down Expand Up @@ -750,6 +750,7 @@ def ewise_mult(self, other, op=binary.times):

def _ewise_mult(self, other, op=binary.times, is_infix=False):
method_name = "ewise_mult"
given = other
if is_infix:
from .infix import ScalarEwiseMultExpr

Expand Down Expand Up @@ -778,14 +779,17 @@ def _ewise_mult(self, other, op=binary.times, is_infix=False):
)
op = get_typed_op(op, self.dtype, other.dtype, kind="binary")
self._expect_op(op, ("BinaryOp", "Monoid"), within=method_name, argname="op")
return ScalarExpression(
expr = ScalarExpression(
method_name,
f"GrB_Vector_eWiseMult_{op.opclass}",
[self._as_vector(), other._as_vector()],
op=op,
is_cscalar=False,
scalar_as_vector=True,
)
if output_type(given) is not Scalar:
expr._right_literal = given # a literal, as Scalar comparisons give it
return expr

def ewise_union(self, other, op, left_default, right_default):
"""Perform element-wise computation on the union of sparse values,
Expand Down Expand Up @@ -940,7 +944,7 @@ def apply(self, op, right=None, *, left=None):

"""
expr = self._as_vector().apply(op, right, left=left)
return ScalarExpression(
rv = ScalarExpression(
expr.method_name,
expr.cfunc_name,
expr.args,
Expand All @@ -950,6 +954,8 @@ def apply(self, op, right=None, *, left=None):
is_cscalar=False,
scalar_as_vector=True,
)
rv._right_literal = expr._right_literal
return rv

def select(self, op, thunk=None):
expr = self._as_vector().select(op, thunk)
Expand Down Expand Up @@ -1137,8 +1143,11 @@ def dup(self, dtype=None, *, clear=False, is_cscalar=False, name=None, **opts):
# End auto-generated code: Scalar


def _scalar_eq(scalar, other):
"""Return ``scalar == other`` as a bool, by ``binary.eq`` (see ``Scalar.__eq__``)."""
def _scalar_eq(scalar, other, within):
"""Return ``scalar == other`` as a bool, by ``binary.eq`` (see ``Scalar.__eq__``).

``within`` is the method an error names, ``__eq__`` or ``__ne__``.
"""
if other is None:
return scalar._is_empty
if type(other) is not Scalar and output_type(other) is Scalar:
Expand Down Expand Up @@ -1175,6 +1184,19 @@ def _scalar_eq(scalar, other):
# float16) holds one value, which numpy compares as binary.eq does, with no
# GraphBLAS call; and numpy compares float16, which Numba cannot.
return _udt_eq_without_numba(scalar, other)
if type(other) is not Scalar:
# As ewise_mult converts it, but an error names the method the user called.
value, dtype = _literal_operand(scalar.dtype, other, binary.eq)
try:
other = Scalar.from_value(value, dtype, is_cscalar=False, name="s_eq_other")
except TypeError:
other = scalar._expect_type(
other,
Scalar,
within=within,
argname="other",
extra_message="Literal scalars also accepted.",
)
return bool(scalar.ewise_mult(other, binary.eq).new(name="s_eq").value)


Expand Down
Loading
Loading