Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/user_guide/init.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,74 @@ raised. However, if the arguments do not match, an error will be raised.

There is no mechanism to change the initialization arguments after graphblas has been
initialized the first time.

.. _suitesparse_jit:

The SuiteSparse:GraphBLAS JIT
-----------------------------

SuiteSparse:GraphBLAS ships precompiled kernels for common operations. For the rest it
can compile a kernel with a C compiler (its JIT), or else run a generic kernel that is
often several times slower (``Matrix.reduce_rowwise`` with ``lor`` on ``BOOL`` or
``plus`` on ``UINT64`` ran 2.7 to 8 times slower on 10 million entries). The JIT control,
``gb.ss.config["jit_c_control"]``, is ``"run"`` by default: run only kernels already
loaded in the process, and compile nothing. UDT operators, and types and operators
registered from C source, raise this default to ``"on"`` when they need a kernel (see
:doc:`udt`), but leave a control you set as it is. To opt in for everything, set the
control to ``"on"`` (compile, load and run):

.. code-block:: bash

export GRAPHBLAS_JIT_C_CONTROL=on

.. code-block:: python

import graphblas as gb

gb.config.set(jit_c_control="on") # before graphblas initializes
gb.ss.config["jit_c_control"] = "on" # or later, for the rest of the process

``graphblas.config["jit_c_control"]`` is read once, when graphblas initializes. It can be
``"on"``, ``"load"`` (load and run cached kernels, never compile), ``"run"``, ``"pause"``
(run no JIT kernels) or ``"off"`` (run none, and unload them). Set it with the
``GRAPHBLAS_JIT_C_CONTROL`` environment variable, with ``gb.config.set`` before graphblas
initializes, or in a YAML file in ``~/.config/graphblas/``. Left unset, it is the
default ``"run"``, which UDT work raises. With nothing configured, a ``"run"`` written
later to ``gb.ss.config`` is that same default, so to keep UDT work from compiling, set
``"run"`` in one of those three ways, or write ``"load"`` or ``"off"`` later. With
``"on"``, python-graphblas checks the C compiler by compiling a tiny type (once per cache
directory and compiler), and falls back to ``"run"``, with no error, where the JIT cannot
work: SuiteSparse built without the JIT (as in the PyPI wheels), SuiteSparse 7, no working
C compiler (in an activated conda environment, ``conda install c-compiler``), or a cache
directory it cannot write to. On Windows, SuiteSparse built with MSVC (as conda-forge
builds it) compiles JIT kernels only through cmake, and python-graphblas does not turn the
JIT on there yet, so a configured ``"on"`` becomes ``"run"``.

With the JIT on:

- **Do not share one cache directory between processes running at the same time** on
SuiteSparse 10.1.1 or later, which no longer locks the cache. They race when they
compile the same new kernel, and calls fail with ``JitError``, leave their outputs
invalid or cleared, return wrong values with no error, or hang
(`GraphBLAS issue #449 <https://github.com/DrTimothyAldenDavis/GraphBLAS/issues/449>`_).
With multiprocessing, dask, pytest-xdist and the like, set
``GRAPHBLAS_JIT_C_CONTROL=run`` in the environment the workers start in, so that none
of them compiles: UDT work raises only the default ``"run"``, not one you set. That
costs UDTs their compiled kernels: UDT operators call their Numba functions instead
(typically 2 to 3 times slower for elementwise operations), and registering types or
operators from C source fails. To keep the kernels, give each process its own cache
instead, by setting the ``GRAPHBLAS_CACHE_PATH`` environment variable before graphblas
initializes. The default will change once SuiteSparse publishes compiled kernels
atomically.
- The first use of each new kernel compiles it, typically in 0.2 to 1 second. Kernels are
cached on disk, in ``gb.ss.config["jit_cache_path"]`` (by default under
``~/.SuiteSparse`` on Linux and macOS), so later runs load them instead.
- The cache is keyed only on the SuiteSparse:GraphBLAS version, so two builds of one
version (a conda build and a source build, say) can load each other's kernels, and get
wrong results or errors if their internals differ. Give each build its own cache path.

If you hit a bug, freeze or crash that may involve the JIT, set
``GRAPHBLAS_JIT_C_CONTROL=run`` (which UDT work leaves alone) and delete the cache
directory, then try again. Please
`report it <https://github.com/python-graphblas/python-graphblas/issues>`_ either way. The
``suitesparse-vanilla`` backend leaves the control as the SuiteSparse build sets it.
27 changes: 17 additions & 10 deletions docs/user_guide/udt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -442,25 +442,32 @@ but never compiles a ``.dylib`` or ``.so``, and silently falls back to the
cfunc path. The 2-3x JIT speedup is silently lost.

python-graphblas repairs the compiler path when ``graphblas.ss`` is
imported, which the first access to ``gb.ss`` does. If
``jit_c_compiler_name`` doesn't exist on disk, it is replaced with one from
``$CONDA_PREFIX/bin/`` (or from ``sysconfig`` for pure-pip installs). The
imported, which the first access to ``gb.ss`` does (and at init, when the JIT
control is configured to ``'on'``). If ``jit_c_compiler_name`` doesn't exist on
disk, it is replaced with one from ``$CONDA_PREFIX/bin/`` (or from
``sysconfig`` for pure-pip installs). The
import changes nothing else, so reading ``gb.ss.about`` does not change what
later operations compute.

``jit_c_control`` is raised from the SS default ``'run'`` (run cached kernels
only; no compile, no load from disk) to ``'on'`` (compile, load, and run) by
each of these, since each asks SuiteSparse to compile C source:
``jit_c_control`` starts at ``graphblas.config["jit_c_control"]``, ``'run'`` by
default (run cached kernels only; no compile, no load from disk); see
:ref:`suitesparse_jit`. That default ``'run'`` is raised to ``'on'`` (compile,
load, and run) by each of these, since each asks SuiteSparse to compile C source:

- a UDT operator that gets C source, such as the first ``binary.plus[udt]``;
- ``gb.dtypes.ss.register_new``, which registers a type from a C typedef;
- ``gb.binary.ss.register_new`` and its ``unary``, ``indexunary``, ``select``,
and ``indexbinary`` siblings, which register an operator from a C definition.

A setting other than ``'run'`` is kept, including ``'off'``, ``'pause'``, and
``'load'`` (load and run cached kernels, never compile). SuiteSparse itself
sets ``'load'`` after a compile fails, so keeping it means a failing compile
is tried once, not again at every later request.
A control you set is kept: ``'off'``, ``'pause'``, ``'load'`` (load and run
cached kernels, never compile), and ``'run'`` too when it was set before init
(``GRAPHBLAS_JIT_C_CONTROL``, ``gb.config.set``, or a YAML config file). With
nothing configured, a ``'run'`` set later in ``gb.ss.config`` is the default
again. Under a ``'run'`` you set, UDT operators use their Numba functions, and
the registrations above fail with an error saying how to opt in:
``gb.ss.config["jit_c_control"] = "on"``, or ``None`` for the default.
SuiteSparse itself sets ``'load'`` after a compile fails, so keeping it means a
failing compile is tried once, not again at every later request.

Call the helper manually to re-fix or verify::

Expand Down
32 changes: 32 additions & 0 deletions graphblas/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,9 @@ def _init(backend_arg, blocking, automatic=False):
# Already initialized with these parameters; nothing more to do
return

if backend_arg == "suitesparse":
# Checked before anything is initialized, so a bad value changes nothing
jit_c_control = _jit_c_control_config()
backend = backend_arg
if backend in {"suitesparse", "suitesparse-vanilla"}:
try:
Expand Down Expand Up @@ -197,6 +200,35 @@ class Lib:
core.ffi = ffi
core.lib = lib
core.NULL = ffi.NULL
if backend == "suitesparse":
from .core.ss.jit_config import _set_jit_c_control_at_init

_set_jit_c_control_at_init(jit_c_control)


_JIT_C_CONTROLS = ("on", "load", "run", "pause", "off")


def _jit_c_control_config():
"""``config["jit_c_control"]``, the SuiteSparse JIT control set at init (see graphblas.yaml).

None when it is not set, which is the default: ``'run'``, raised by UDT work.
"""
val = config.get("jit_c_control")
if val is None:
return None
# YAML 1.1 reads an unquoted on/off as a bool, so a config file saying
# ``jit_c_control: off`` arrives here as False.
if val is True or val is False:
return "on" if val else "off"
if isinstance(val, str) and val.lower() in _JIT_C_CONTROLS:
return val.lower()
raise ValueError(
"Bad value for graphblas.config['jit_c_control'] (or the GRAPHBLAS_JIT_C_CONTROL "
f"environment variable): {val!r}. Must be one of: "
+ ", ".join(repr(x) for x in _JIT_C_CONTROLS)
+ ", or unset"
)


# Ideally this is in operator.py, but lives here to avoid circular references
Expand Down
29 changes: 22 additions & 7 deletions graphblas/core/operator/binary.py
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,7 @@ def _pair_dtype(op, dtype, dtype2):
_has_jit_set,
_maybe_warn_jit_skipped,
compile_udt_binary_wrapper,
set_jit_c_mixed_on_op,
set_jit_c_on_op,
)

Expand Down Expand Up @@ -689,13 +690,13 @@ def _compile_udt(self, dtype, dtype2):
op = _finalize_udt_op(
self, dtype, dtype2, ret_type, binary_wrapper, wrapper_sig, TypedUserBinaryOp
)
# Set JIT C definition for auto-generated built-in UDT ops. Only
# same-type pairs apply: mixed UDT+scalar ops don't have a
# meaningful single-type C definition. The arithmetic kernel also
# declares ``z`` with the operands' type, so it fits only an op whose
# result is that type too, which ``truediv`` on integers is not.
# ``_has_jit_set`` gates access to ``lib.GrB_BinaryOp_set_String``,
# which is absent on SS < 9.
# Set JIT C definition for auto-generated built-in UDT ops. A same-type
# pair whose result is that type too gets the one-type kernel below; a
# mixed pair (a scalar with a UDT, or two UDTs of one layout with other
# element types) gets a kernel with a C type for each of z, x and y,
# when it has a C form (``set_jit_c_mixed_on_op``); eq/ne on a mixed
# pair keep the cfunc. ``_has_jit_set`` gates access to
# ``lib.GrB_BinaryOp_set_String``, which is absent on SS < 9.
if is_lifted and self.name in _BUILTIN_UDT_BINARY_OPS:
is_jittable = ret_type == dtype
else:
Expand Down Expand Up @@ -723,6 +724,20 @@ def _compile_udt(self, dtype, dtype2):
is_eq=(self.name == "eq"),
)
_maybe_warn_jit_skipped(op._jit_c_info, self.name, dtype.name)
elif _has_numba and _has_jit_set and is_lifted and self.name in _BUILTIN_UDT_BINARY_OPS:
# (no warning when there is none: the cfunc is this op's documented path)
from ..udt import _udt_op_types

_, input_types = _udt_op_types(self.name, dtype, dtype2)
op._jit_c_info = set_jit_c_mixed_on_op(
op.gb_obj,
self.name,
_BUILTIN_UDT_BINARY_OPS[self.name],
dtype,
dtype2,
ret_type,
input_types,
)
return op

@classmethod
Expand Down
127 changes: 124 additions & 3 deletions graphblas/core/operator/udt_codegen.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"""

import ast
import hashlib
import itertools
import linecache
from functools import lru_cache, reduce
Expand Down Expand Up @@ -1419,6 +1420,102 @@ def _make_jit_c_definition(op_name, py_op, dtype, arity):
return c_name, f"void {c_name} ({params}) {{ {assigns} }}"


def _c_type_name(dtype):
"""The C name SuiteSparse knows ``dtype`` by: a UDT's pinned JIT name, or the C type of
a built-in scalar dtype. ``None`` when it has none.
"""
if dtype.np_type.names is None and dtype.np_type.subdtype is None:
return NP_TO_C_TYPES.get(dtype.np_type)
typedef_info = _udt_c_typedef(dtype.jit_c_name or dtype.name, dtype.np_type)
return None if typedef_info is None else typedef_info[0]


def _c_operand(side, path, stored, computed):
"""C source reading operand ``side`` (``*x``, or the element at ``path``) in the type the
op computes in. A conversion in C (integer to floating, a real to complex, a narrower
type to a wider one) gives what numpy's ``astype`` gives, which the cfunc uses.
"""
expr = f"(*{side})" if path is None else f"{side}->{path}"
if stored == computed:
return expr
return f"(({NP_TO_C_TYPES[computed]}) ({expr}))"


def _make_jit_c_mixed_definition(op_name, py_op, dtype, dtype2, ret_type, input_types):
"""Generate a JIT C function definition for a lifted op whose operand types differ, or
whose result type differs from them: a scalar with an array or record UDT (either
order), two array UDTs of one shape with different element types, two records with the
same fields and different leaf types.

The signature has a C type for each of z, x and y, so the C name carries all three. A
scalar operand is read as ``(*x)`` for every element or leaf. Returns ``(c_name,
c_defn)``, or ``None`` when one of the types has no C name, when an array operand
broadcasts (another shape than the result's), when a record has an array leaf, or when
an element is not computed in the result's element type (integer ``truediv`` computes
in its operands' type and returns float); those keep the Numba cfunc.

``input_types`` is what :func:`udt._udt_op_types` returns with ``ret_type``: each
element is computed as the built-in op on the operands' element types computes it,
after converting both operands to that type, as the cfunc does.
"""
names = [_c_type_name(t) for t in (ret_type, dtype, dtype2)]
if None in names:
return None
z_name, x_name, y_name = names
np_x, np_y, np_z = dtype.np_type, dtype2.np_type, ret_type.np_type

def assign(target, x_path, y_path, x_stored, y_stored, types, out):
in_x, in_y = types
if not in_x == in_y == out:
return None
lhs = _c_operand("x", x_path, x_stored, in_x)
rhs = _c_operand("y", y_path, y_stored, in_y)
return _c_assign_binary(py_op, target, lhs, rhs, out)

if np_z.names is not None:
leaves_z = list(_iter_record_leaves(np_z))
leaves_x = list(_iter_record_leaves(np_x)) if np_x.names is not None else None
leaves_y = list(_iter_record_leaves(np_y)) if np_y.names is not None else None
stmts = []
for i, (_py, c, leaf) in enumerate(leaves_z):
# A record with an array leaf has no C type name, so it returned above
if leaf.shape: # pragma: no cover (safety)
return None
x_path, x_stored = (None, np_x) if leaves_x is None else (c, leaves_x[i][2])
y_path, y_stored = (None, np_y) if leaves_y is None else (c, leaves_y[i][2])
stmt = assign(f"z->{c}", x_path, y_path, x_stored, y_stored, input_types[i], leaf)
if stmt is None:
return None
stmts.append(stmt)
body = " ".join(stmts)
else:
base_z, shape_z = np_z.subdtype
operands = []
for np_t in (np_x, np_y):
if np_t.subdtype is None:
operands.append((None, np_t))
elif np_t.subdtype[1] != shape_z:
return None # (broadcasting reads through an index map)
else:
operands.append(("v[i]", np_t.subdtype[0]))
(x_path, x_stored), (y_path, y_stored) = operands
stmt = assign("z->v[i]", x_path, y_path, x_stored, y_stored, input_types[0], base_z)
if stmt is None:
return None
body = f"for (int64_t i = 0 ; i < {reduce(mul, shape_z)} ; i++) {{ {stmt} }}"
# The length of x's name keeps two pairs from sharing a C name ("A" and "B_C", "A_B" and
# "C"), and keeps it from any one-type name (``{op_name}_{type}``, whose type name cannot
# start with a digit). SuiteSparse finds a compiled kernel by its op's name, so two ops
# with one name would run each other's kernel.
c_name = f"{op_name}_{len(x_name)}_{x_name}_{y_name}"
if len(c_name) >= lib.GxB_MAX_NAME_LEN:
# SuiteSparse keeps at most 127 characters (see ``_set_jit_c_strings``). A length of 0
# cannot come from the readable form, so this name is still the pair's own.
c_name = f"{op_name}_0_{hashlib.sha256(c_name.encode()).hexdigest()[:32]}"
params = f"{z_name} *z, const {x_name} *x, const {y_name} *y"
return c_name, f"void {c_name} ({params}) {{ {body} }}"


def _make_jit_c_comparison_definition(op_name, dtype, *, is_eq):
"""Generate a JIT C function definition for ``binary.eq[udt]`` /
``binary.ne[udt]``.
Expand Down Expand Up @@ -1499,8 +1596,15 @@ def _set_jit_c_strings(gb_obj, c_name, c_defn, set_string_func):
from ..ss.jit_config import _enable_jit_for_udt

_enable_jit_for_udt()
set_string_func(gb_obj, ffi.new("char[]", c_name.encode()), lib.GxB_JIT_C_NAME)
# SuiteSparse refuses a name of GxB_MAX_NAME_LEN (128) characters or more, then takes the
# definition under the empty name: its kernels call no function and leave their outputs
# uninitialized. So the definition goes only with the name, and without it the op keeps
# its cfunc.
info = set_string_func(gb_obj, ffi.new("char[]", c_name.encode()), lib.GxB_JIT_C_NAME)
if info != lib.GrB_SUCCESS:
return False
set_string_func(gb_obj, ffi.new("char[]", c_defn.encode()), lib.GxB_JIT_C_DEFINITION)
return True


def set_jit_c_comparison_on_op(gb_obj, op_name, dtype, set_string_func, *, is_eq):
Expand All @@ -1516,7 +1620,23 @@ def set_jit_c_comparison_on_op(gb_obj, op_name, dtype, set_string_func, *, is_eq
result = _make_jit_c_comparison_definition(op_name, dtype, is_eq=is_eq)
if result is None:
return None
_set_jit_c_strings(gb_obj, *result, set_string_func)
if not _set_jit_c_strings(gb_obj, *result, set_string_func):
return None
return result


def set_jit_c_mixed_on_op(gb_obj, op_name, py_op, dtype, dtype2, ret_type, input_types):
"""Generate and set the JIT C name and definition of a lifted binary op on two types
(:func:`_make_jit_c_mixed_definition`). Returns ``(c_name, c_defn)``, or ``None`` when
the op has no C form (it keeps the Numba cfunc). Callers check ``_has_jit_set`` first,
as for :func:`set_jit_c_on_op`: without it, SuiteSparse has no
``GrB_BinaryOp_set_String``.
"""
result = _make_jit_c_mixed_definition(op_name, py_op, dtype, dtype2, ret_type, input_types)
if result is None:
return None
if not _set_jit_c_strings(gb_obj, *result, lib.GrB_BinaryOp_set_String):
return None
return result


Expand All @@ -1532,5 +1652,6 @@ def set_jit_c_on_op(gb_obj, op_name, py_op, dtype, set_string_func, arity=2):
result = _make_jit_c_definition(op_name, py_op, dtype, arity)
if result is None:
return None
_set_jit_c_strings(gb_obj, *result, set_string_func)
if not _set_jit_c_strings(gb_obj, *result, set_string_func):
return None
return result
Loading
Loading