diff --git a/docs/user_guide/init.rst b/docs/user_guide/init.rst index 141e1f250..306b596a4 100644 --- a/docs/user_guide/init.rst +++ b/docs/user_guide/init.rst @@ -69,8 +69,9 @@ often several times slower (``Matrix.reduce_rowwise`` with ``lor`` on ``BOOL`` o ``plus`` on ``UINT64`` ran 2.7 to 8 times slower on 10 million entries). The JIT control, ``gb.ss.config["jit_c_control"]``, is ``"run"`` by default: run only kernels already loaded in the process, and compile nothing. UDT operators, and types and operators -registered from C source, raise this default to ``"on"`` when they need a kernel (see -:doc:`udt`), but leave a control you set as it is. To opt in for everything, set the +registered from C source, raise this default to ``"on"`` the first time they need a +kernel, for the rest of the process and for every operation (see :doc:`udt`), but leave +a control you set as it is. To opt in for everything, set the control to ``"on"`` (compile, load and run): .. code-block:: bash @@ -96,7 +97,11 @@ later to ``gb.ss.config`` is that same default, so to keep UDT work from compili directory and compiler), and falls back to ``"run"``, with no error, where the JIT cannot work: SuiteSparse built without the JIT (as in the PyPI wheels), SuiteSparse 7, no working C compiler (in an activated conda environment, ``conda install c-compiler``), or a cache -directory it cannot write to. On Windows, SuiteSparse built with MSVC (as conda-forge +directory it cannot write to. That check is for a configured ``"on"`` only: an ``"on"`` +written later to ``gb.ss.config`` is taken as it is, and with a C compiler that does not +work, the first operation that needs a new kernel raises ``JitError``, and SuiteSparse +lowers the control to ``"load"``. ``gb.ss.fix_jit_config()`` repairs the compiler setting +where it can and checks it. On Windows, SuiteSparse built with MSVC (as conda-forge builds it) compiles JIT kernels only through cmake, and python-graphblas does not turn the JIT on there yet, so a configured ``"on"`` becomes ``"run"``. @@ -109,8 +114,10 @@ With the JIT on: (`GraphBLAS issue #449 `_). With multiprocessing, dask, pytest-xdist and the like, set ``GRAPHBLAS_JIT_C_CONTROL=run`` in the environment the workers start in, so that none - of them compiles: UDT work raises only the default ``"run"``, not one you set. That - costs UDTs their compiled kernels: UDT operators call their Numba functions instead + of them compiles: UDT work raises only the default ``"run"``, not one you set. (At the + default, the first UDT operator that needs a kernel turns the JIT on for the whole + process, so such processes race too.) Setting ``"run"`` costs UDTs their compiled + kernels: UDT operators call their Numba functions instead (typically 2 to 3 times slower for elementwise operations), and registering types or operators from C source fails. To keep the kernels, give each process its own cache instead, by setting the ``GRAPHBLAS_CACHE_PATH`` environment variable before graphblas diff --git a/docs/user_guide/udt.rst b/docs/user_guide/udt.rst index 96d8e1f91..88b77086a 100644 --- a/docs/user_guide/udt.rst +++ b/docs/user_guide/udt.rst @@ -376,9 +376,11 @@ from Numba's typing pass instead of a 200-line traceback. JIT and introspection --------------------- -When the UDT name and all field names are valid C identifiers (not C reserved -words), and the field types map to C primitives, SuiteSparse JIT-compiles a -dedicated C kernel for each ``op[udt]`` lookup. The kernel is cached on disk +When all field names are valid C identifiers (not C reserved words), and the +field types map to C primitives, SuiteSparse JIT-compiles a dedicated C kernel +for each ``op[udt]`` lookup. A lifted op on a UDT and a built-in type, or on two +UDT types that pair, compiles too, under a name such as +``plus_7_int64_t_gbudt_Edge``. The kernel is cached on disk and reused across processes (keyed by content hash, so renaming a UDT doesn't invalidate). JIT lets SuiteSparse inline the kernel into its eWise and reduce templates, eliminating the per-element function-call overhead the Numba @@ -389,11 +391,11 @@ Inspect what SuiteSparse is JIT-ing from Python: .. code-block:: python - udt.jit_c_name # 'Edge', or None if not JIT-able - udt.jit_c_definition # 'typedef struct { ... } Edge ;' + udt.jit_c_name # 'gbudt_Edge', or None if not JIT-able + udt.jit_c_definition # 'typedef struct { ... } gbudt_Edge ;' op = binary.plus[udt] - op.jit_c_name # 'plus_Edge' + op.jit_c_name # 'plus_gbudt_Edge' op.jit_c_source # full C source SS will compile monoid.plus[udt].jit_c_source # same kernel @@ -404,10 +406,11 @@ Inspect what SuiteSparse is JIT-ing from Python: JIT is skipped when: -- A field name (or the UDT name, if you supplied one) is a C reserved word - (``class``, ``return``, ...) or a stdlib macro/typedef pulled in by - ``GraphBLAS.h`` (``NULL``, ``FILE``, ``M_PI``, ``complex``, ...). The op - falls through to the Numba cfunc path. +- A field name is a C reserved word (``class``, ``return``, ...), a macro or + typedef of the C headers the kernels include (``NULL``, ``FILE``, ``M_PI``, + ``complex``, ``INT_MAX``, ``SIZE_MAX``, ``BUFSIZ``, ``SIGINT``, ...), or + starts with ``GB_``, ``GrB_`` or ``GxB_``, which GraphBLAS keeps for its own + names. The op falls through to the Numba cfunc path. - A field type isn't in the numpy-to-C map (rare; the standard numeric scalar types all map). - A record has an array-valued field. Built-in operators still lift to it @@ -418,11 +421,25 @@ JIT is skipped when: Pass ``align=True`` to ``np.dtype``, or use the dict / dataclass form, which auto-aligns. - The JIT compiler isn't usable after auto-fix (see below). - -Anonymous UDTs (no ``name=`` argument) still take the JIT path: a synthetic -``_gbudt_NNN`` C name is minted so SuiteSparse always has a registerable -identifier. Pass ``name=`` only for readable JIT cache filenames and -introspection. +- An op on a UDT and another type is an array UDT that broadcasts to another + shape, a record with an array-valued field, or ``eq`` or ``ne``; or an op on + one UDT gives another type (``v / v`` on ``INT8[3]`` is ``FP64[3]``). These + keep the Numba cfunc without a warning. + +SuiteSparse knows a UDT by ``gbudt_`` and its name, which cannot collide with +an identifier in the C code of a kernel, whatever the name. A name starting +with ``_gb``, which python-graphblas keeps for the types it makes itself, is +the exception: it is the C name as it is. A UDT without a name, or whose name +is not a C identifier or would make that C name too long to leave room for its +ops' names (over 118 characters), gets a synthetic ``_gbudt_NNN`` C name and +still takes the JIT path. Pass ``name=`` only for +readable JIT cache filenames and introspection. To name such a UDT in C source +of your own, as for ``gb.binary.ss.register_new``, use its ``jit_c_name``, not +its Python name, which SuiteSparse does not know. A type registered from C +source (``gb.dtypes.ss.register_new``) keeps the name you give it, so give it +118 characters or fewer: the built-in operators lifted to it are named with +that name after them (``floordiv_``, ``plus_``, ...), and a longer one leaves +them names SuiteSparse refuses, so they run without the JIT. When the cause is the UDT itself (the first four cases above), the first time auto-lift produces a non-JIT'd op for a given ``(op, dtype)`` pair in a @@ -439,7 +456,9 @@ category or by message:: warnings.filterwarnings("ignore", message="UDT operator running without JIT") In every skipped case the operation still works correctly via the Numba -function-pointer path; you only lose the JIT speedup. +function-pointer path; you only lose the JIT speedup. A field named like a +macro that only some platform's C headers define (beyond the C standard's) +is not skipped, and its kernel fails to compile; rename the field. JIT compiler auto-fix ~~~~~~~~~~~~~~~~~~~~~ @@ -461,9 +480,12 @@ later operations compute. ``jit_c_control`` starts at ``graphblas.config["jit_c_control"]``, ``'run'`` by default (run cached kernels only; no compile, no load from disk); see :ref:`suitesparse_jit`. That default ``'run'`` is raised to ``'on'`` (compile, -load, and run) by each of these, since each asks SuiteSparse to compile C source: +load, and run) by each of these, since each asks SuiteSparse to compile C source. +It stays ``'on'`` for the rest of the process, for every operation, built-in ones +included: -- a UDT operator that gets C source, such as the first ``binary.plus[udt]``; +- a UDT operator that gets C source, such as the first ``binary.plus[udt]`` or + ``binary.plus[udt, "INT64"]``; - ``gb.dtypes.ss.register_new``, which registers a type from a C typedef; - ``gb.binary.ss.register_new`` and its ``unary``, ``indexunary``, ``select``, and ``indexbinary`` siblings, which register an operator from a C definition. diff --git a/graphblas/core/dtypes.py b/graphblas/core/dtypes.py index 2a85f46a1..97cbf1f62 100644 --- a/graphblas/core/dtypes.py +++ b/graphblas/core/dtypes.py @@ -108,14 +108,14 @@ def _deserialize(name, dtype, is_anonymous): def _get_udt_c_type_info(datatype): """Return ``(c_name, c_typedef)`` for a UDT, or ``None`` if not expressible in C. - The returned ``c_name`` may differ from ``datatype.name``: if the user - didn't supply a name, or supplied one that isn't a valid C identifier, - ``_udt_c_typedef`` synthesizes a fresh ``_gbudt_NNN`` name so the UDT - can still take the JIT path. Returns ``None`` only for the cases that - remain unrepresentable in C: an unsupported field type (object, - datetime, string), or a field name that collides with a C reserved - word or isn't an identifier. Supports record UDTs (including nested - records) and array UDTs (e.g., ``FP64[3]``). + The returned ``c_name`` is ``gbudt_`` and ``datatype.name``, or a fresh + ``_gbudt_NNN`` name when the user didn't supply a name, or supplied one + that isn't a valid C identifier or is too long for SuiteSparse + (``udt_codegen._pick_c_type_name``), so the UDT can still take the JIT + path. Returns ``None`` only for the cases that remain unrepresentable in + C: an unsupported field type (object, datetime, string), or a field name + that collides with a C reserved word or isn't an identifier. Supports + record UDTs (including nested records) and array UDTs (e.g., ``FP64[3]``). """ from .operator.udt_codegen import _udt_c_typedef @@ -159,7 +159,8 @@ def _set_udt_jit_c_definition(datatype): # A name already given to another layout (two UDTs registered under one # name) gets a fresh one: SuiteSparse would share their JIT kernels. while _jit_c_type_defs.setdefault(c_name, typedef) != typedef: - info = _udt_c_typedef(f"_gbudt_{next(_synthetic_udt_counter)}", datatype.np_type) + c_name = f"_gbudt_{next(_synthetic_udt_counter)}" + info = _udt_c_typedef(c_name, datatype.np_type, is_c_name=True) c_name, typedef = info lib.GrB_Type_set_String(datatype._carg, ffi.new("char[]", c_name.encode()), lib.GxB_JIT_C_NAME) lib.GrB_Type_set_String( @@ -276,9 +277,9 @@ def register_anonymous(dtype, name=None): string, and nested-UDT fields are not. name : str, optional A human-readable name for this UDT. Used in error messages and - ``repr``. Must be a valid C identifier (and not a C reserved word) - to be usable on the SuiteSparse JIT path; otherwise JIT is skipped - silently and ops fall back to the Numba cfunc path. + ``repr``. Any name takes the SuiteSparse JIT path: its C name is + ``gbudt_`` and the name, or a synthetic one when that is not a C + identifier or is too long. Returns ------- diff --git a/graphblas/core/operator/udt_codegen.py b/graphblas/core/operator/udt_codegen.py index 41fba757a..0ac54404a 100644 --- a/graphblas/core/operator/udt_codegen.py +++ b/graphblas/core/operator/udt_codegen.py @@ -147,25 +147,20 @@ def _is_signed_int(np_dtype): # ``hasattr`` would lie; gate on the backend too. _has_jit_set = backend == "suitesparse" and hasattr(lib, "GxB_JIT_C_NAME") -# Names that can't be used as UDT type names or record field names in the -# JIT C source. Covers two failure modes from the JIT include chain -# (``GraphBLAS.h`` transitively pulls in ````, ````, -# ````, ````, ````, ````): +# Names that can't be used as record field names in the JIT C source: C +# keywords, and the macros of the JIT include chain (``GraphBLAS.h`` +# transitively pulls in ````, ````, ````, +# ````, ````, ```` and more). The preprocessor +# expands a macro inside any declarator or expression it appears in, so a field +# declared ``double M_PI ;`` becomes ``double 3.14159... ;`` and won't compile; +# the separate namespace for struct members doesn't help. (A UDT's own name +# never collides: its C name has a prefix, ``_C_TYPE_NAME_PREFIX``. The typedef +# names listed here mattered when the type's C name was its Python name.) # -# 1. **Macro names**: the preprocessor expands them inside any declarator -# or expression they appear in, so a field declared ``double M_PI ;`` -# becomes ``double 3.14159... ;`` and won't compile. The C-standard -# separate namespace for struct members doesn't help here. -# 2. **Typedef names** at the *outer* type-name position: emitting -# ``typedef struct { ... } FILE ;`` collides with the existing -# ``FILE`` typedef in scope (most compilers reject as a redefinition). -# Struct member names of the same spelling are fine (members live in -# their own namespace), but our gate runs the same check at both -# positions for simplicity. -# -# Either way the JIT compile fails, SuiteSparse swallows the failure -# silently, and the op runs through the slower Numba cfunc path. Block -# these eagerly so the user sees the "without JIT" warning instead. +# Either way the JIT compile fails. SuiteSparse 9.4 and later then raise +# JitError for that call and set ``jit_c_control`` to 'load', so the process +# compiles no more kernels. Block these eagerly: the op runs through the +# slower Numba cfunc path, and the user sees the "without JIT" warning. _C_RESERVED = frozenset( { # C keywords @@ -280,17 +275,80 @@ def _is_signed_int(np_dtype): # "errno", } +) | frozenset( + # Object-like macros the C standard defines in the other headers of the JIT + # include chain. A platform adds its own (POSIX errno codes, ...), which + # still fail to compile. + [ + # + *("CHAR_BIT", "SCHAR_MIN", "SCHAR_MAX", "UCHAR_MAX", "CHAR_MIN", "CHAR_MAX"), + *("MB_LEN_MAX", "SHRT_MIN", "SHRT_MAX", "USHRT_MAX", "INT_MIN", "INT_MAX"), + *("UINT_MAX", "LONG_MIN", "LONG_MAX", "ULONG_MAX", "LLONG_MIN", "LLONG_MAX"), + "ULLONG_MAX", + # + *( + f"{u}INT{kind}{bits}_{end}" + for kind in ("", "_LEAST", "_FAST") + for bits in (8, 16, 32, 64) + for u, end in (("", "MIN"), ("", "MAX"), ("U", "MAX")) + ), + *("INTPTR_MIN", "INTPTR_MAX", "UINTPTR_MAX", "INTMAX_MIN", "INTMAX_MAX"), + *("UINTMAX_MAX", "PTRDIFF_MIN", "PTRDIFF_MAX", "SIG_ATOMIC_MIN", "SIG_ATOMIC_MAX"), + *("SIZE_MAX", "WCHAR_MIN", "WCHAR_MAX", "WINT_MIN", "WINT_MAX"), + # + *( + f"{prefix}{conv}{size}" + for prefix, convs in (("PRI", "diouxX"), ("SCN", "dioux")) + for conv in convs + for size in ( + *(f"{kind}{bits}" for kind in ("", "LEAST", "FAST") for bits in (8, 16, 32, 64)), + "MAX", + "PTR", + ) + ), + # + *("BUFSIZ", "FILENAME_MAX", "FOPEN_MAX", "L_tmpnam", "SEEK_CUR", "SEEK_END"), + *("SEEK_SET", "TMP_MAX", "_IOFBF", "_IOLBF", "_IONBF"), + # + *("EXIT_FAILURE", "EXIT_SUCCESS", "RAND_MAX", "MB_CUR_MAX"), + # + *("EDOM", "ERANGE", "EILSEQ"), + # + *("FP_INFINITE", "FP_NAN", "FP_NORMAL", "FP_SUBNORMAL", "FP_ZERO", "FP_FAST_FMA"), + *("FP_FAST_FMAF", "FP_FAST_FMAL", "FP_ILOGB0", "FP_ILOGBNAN", "MATH_ERRNO"), + *("MATH_ERREXCEPT", "math_errhandling"), + # + *( + f"ATOMIC_{kind}_LOCK_FREE" + for kind in ("BOOL", "CHAR", "CHAR16_T", "CHAR32_T", "WCHAR_T", "SHORT", "INT") + ), + *("ATOMIC_LONG_LOCK_FREE", "ATOMIC_LLONG_LOCK_FREE", "ATOMIC_POINTER_LOCK_FREE"), + "ATOMIC_FLAG_INIT", + # , which the C library headers bring in on some platforms + *("SIGABRT", "SIGFPE", "SIGILL", "SIGINT", "SIGSEGV", "SIGTERM"), + *("SIG_DFL", "SIG_ERR", "SIG_IGN"), + ] ) +# GraphBLAS's own names, many of them macros of ``GraphBLAS.h`` and its JIT headers +# (``GrB_INDEX_MAX``, ``GxB_IMPLEMENTATION_MAJOR``), and more with each version, so a +# field may not start with any of these. +_C_RESERVED_PREFIXES = ("GB_", "GrB_", "GxB_") + def _is_valid_c_identifier(name): - """True if ``name`` is a valid C identifier and not a reserved word.""" - return isinstance(name, str) and name.isidentifier() and name not in _C_RESERVED + """True if ``name`` is a valid C identifier and not a reserved word or GraphBLAS's name.""" + return ( + isinstance(name, str) + and name.isidentifier() + and name not in _C_RESERVED + and not name.startswith(_C_RESERVED_PREFIXES) + ) # Process-global counter for synthesizing C names when the user-supplied -# Python name isn't a valid C identifier (or is a C reserved word, or the -# UDT was registered without ``name=`` at all). The Python-side ``DataType.name`` +# Python name doesn't make a valid C identifier of 127 characters or fewer (or +# the UDT was registered without ``name=`` at all). The Python-side ``DataType.name`` # is left alone; this only affects ``GxB_JIT_C_NAME``. ``itertools.count`` is # atomic enough for our purposes; collisions across processes are harmless # because the SS JIT cache files are keyed by ``(c_name, content_hash)`` per @@ -304,29 +362,61 @@ def _is_valid_c_identifier(name): _jit_c_type_defs = {} +# The C name of a UDT is its name after this prefix. SuiteSparse's JIT puts the type's name in +# a kernel beside its own identifiers, so a UDT named like one of them did not compile: a +# local of its templates (``A``, ``C``, ``p``, ``i``, ``result``), a parameter of the op +# definitions made here (``z``, ``x``), or anything the C headers declare (``div``, +# ``index``, ``GB_void``). No identifier there starts with the prefix. +_C_TYPE_NAME_PREFIX = "gbudt_" + + +# The ops whose C name is ``{op}_{type}`` (``set_jit_c_on_op``, +# ``set_jit_c_comparison_on_op``): the lifted arithmetic, eq and ne. A mixed op +# shortens its own name (``_make_jit_c_mixed_definition``). +_ONE_TYPE_OP_NAMES = (*BUILTIN_UDT_BINARY_OPS, *BUILTIN_UDT_UNARY_OPS, "eq", "ne") + + +def _max_type_c_name_len(): + """The longest C name a UDT may have, so every ``{op}_{type}`` built from it fits. + + SuiteSparse refuses a name of ``GxB_MAX_NAME_LEN`` (128) characters or more; the + longest op prefix is ``floordiv_``, so this is 118. + """ + return lib.GxB_MAX_NAME_LEN - 1 - max(len(f"{name}_") for name in _ONE_TYPE_OP_NAMES) + + def _pick_c_type_name(python_name): - """Return a valid C identifier for use as ``GxB_JIT_C_NAME`` on a UDT. - - When ``python_name`` is already a valid C identifier (and not reserved), - return it unchanged so introspection and JIT cache filenames remain - readable. Otherwise mint a fresh ``_gbudt_NNN`` name. This lets UDTs - that were anonymous in the Python sense (no ``name=`` passed, name with - dots / special chars, name that collides with a C keyword) still take - the JIT path; users never see the synthetic name unless they read - ``DataType.jit_c_name``. + """Return a valid C identifier for use as ``GxB_JIT_C_NAME`` on a UDT named ``python_name``. + + ``gbudt_`` and the name (see ``_C_TYPE_NAME_PREFIX``), which stays readable in + introspection and JIT cache file names. A name starting with ``_gb``, which + python-graphblas keeps for the types it makes itself (a synthetic ``_gbudt_NNN`` or + ``_gbnest_NNN``, the tensor engine's ``_gbtensor_...``), and which no kernel identifier + starts with, is kept as it is. When the result is not a C identifier (a name with dots, + or none given), or is too long for each op built from it to fit SuiteSparse's + ``GxB_MAX_NAME_LEN`` (``_max_type_c_name_len``), mint a fresh ``_gbudt_NNN`` name + instead, so the UDT and its ops still take the JIT path. A name SuiteSparse already has + for a type goes to ``_udt_c_typedef`` with ``is_c_name`` instead. """ - if _is_valid_c_identifier(python_name): - return python_name + if not isinstance(python_name, str): + c_name = "" + elif python_name.startswith("_gb"): + c_name = python_name + else: + c_name = _C_TYPE_NAME_PREFIX + python_name + if c_name.isidentifier() and len(c_name) <= _max_type_c_name_len(): + return c_name return f"_gbudt_{next(_synthetic_udt_counter)}" -def _udt_c_typedef(python_name, np_type): +def _udt_c_typedef(python_name, np_type, *, is_c_name=False): """Build the JIT C typedef for a record or array UDT, or return ``None``. Returns ``(type_name, typedef)`` on success, where ``type_name`` is the - C identifier we register with SuiteSparse (may differ from - ``python_name`` if that wasn't a valid C identifier; see - ``_pick_c_type_name``). Returns ``None`` when the UDT still can't be + C identifier we register with SuiteSparse (``gbudt_`` and ``python_name``, or + a synthetic name; see ``_pick_c_type_name``). With ``is_c_name``, + ``python_name`` is already that C name: one SuiteSparse has for the type, or a + synthetic one made here. Returns ``None`` when the UDT still can't be expressed in C after synthesizing the top-level name (a field name collides with a C reserved word, or a leaf field type isn't in ``NP_TO_C_TYPES``, e.g. object or datetime). @@ -345,7 +435,7 @@ def _udt_c_typedef(python_name, np_type): # than register a broken typedef; the cfunc path is still correct. if np_type.names is not None and not _is_c_compatible_layout(np_type): return None - type_name = _pick_c_type_name(python_name) + type_name = python_name if is_c_name else _pick_c_type_name(python_name) if np_type.names is not None: # Inner typedefs (for nested structured fields) are prepended to the # final definition string so the outer struct's field types are in @@ -366,7 +456,7 @@ def _udt_c_typedef(python_name, np_type): # The inner C name is synthesized to avoid collisions with # user-supplied type names elsewhere in the process. inner_name = f"_gbnest_{next(_synthetic_udt_counter)}" - inner_info = _udt_c_typedef(inner_name, field_dtype) + inner_info = _udt_c_typedef(inner_name, field_dtype, is_c_name=True) if inner_info is None: return None inner_resolved_name, inner_typedef = inner_info @@ -1374,10 +1464,9 @@ def _make_jit_c_definition(op_name, py_op, dtype, arity): # propagate to SS. If the op's signature referenced ``dtype.name`` it would # use an undefined struct name, and SS would silently fall back to the # Numba cfunc instead of JIT-compiling. ``jit_c_name`` is also where the - # synthetic ``_gbudt_NNN`` name (for UDTs whose Python name isn't a valid - # C identifier) is recorded; using it keeps op codegen consistent. - pinned_name = dtype.jit_c_name - typedef_info = _udt_c_typedef(pinned_name or dtype.name, dtype.np_type) + # ``gbudt_`` or synthetic ``_gbudt_NNN`` name (see ``_pick_c_type_name``) is + # recorded; using it keeps op codegen consistent. + typedef_info = _dtype_c_typedef(dtype) if typedef_info is None: return None type_name, _typedef = typedef_info @@ -1426,10 +1515,22 @@ def _c_type_name(dtype): """ if dtype.np_type.names is None and dtype.np_type.subdtype is None: return NP_TO_C_TYPES.get(dtype.np_type) - typedef_info = _udt_c_typedef(dtype.jit_c_name or dtype.name, dtype.np_type) + typedef_info = _dtype_c_typedef(dtype) return None if typedef_info is None else typedef_info[0] +def _dtype_c_typedef(dtype): + """``_udt_c_typedef`` for UDT ``dtype`` under the C name SuiteSparse has for it. + + That is its ``jit_c_name``: pinned when a UDT from numpy registers, or the name of a + type registered from C source. A UDT with none (not expressible in C) gets the name + ``_pick_c_type_name`` would give it. + """ + if (c_name := dtype.jit_c_name) is not None: + return _udt_c_typedef(c_name, dtype.np_type, is_c_name=True) + return _udt_c_typedef(dtype.name, dtype.np_type) + + def _c_operand(side, path, stored, computed): """C source reading operand ``side`` (``*x``, or the element at ``path``) in the type the op computes in. A conversion in C (integer to floating, a real to complex, a narrower @@ -1542,10 +1643,9 @@ def _make_jit_c_comparison_definition(op_name, dtype, *, is_eq): semantic comparison. """ np_type = dtype.np_type - pinned_name = dtype.jit_c_name # ``_udt_c_typedef`` returns None when the numpy layout doesn't match a # C compiler's layout for the same struct. - typedef_info = _udt_c_typedef(pinned_name or dtype.name, np_type) + typedef_info = _dtype_c_typedef(dtype) if typedef_info is None: return None type_name, _typedef = typedef_info diff --git a/graphblas/core/ss/binary.py b/graphblas/core/ss/binary.py index 547d06ccf..d940ac342 100644 --- a/graphblas/core/ss/binary.py +++ b/graphblas/core/ss/binary.py @@ -61,6 +61,8 @@ def register_new(name, jit_c_definition, left_type, right_type, ret_type): jit_c_definition : str The C definition as a string of the user-defined function. For example: ``"void absdiff (double *z, double *x, double *y) { (*z) = fabs ((*x) - (*y)) ; }"``. + A UDT made from a numpy dtype is named in C by its ``jit_c_name`` (usually ``gbudt_`` + and its name), not its Python name. left_type : dtype The dtype of the left operand of the binary operator. right_type : dtype diff --git a/graphblas/core/ss/dtypes.py b/graphblas/core/ss/dtypes.py index 0ccdd8264..54d3fb812 100644 --- a/graphblas/core/ss/dtypes.py +++ b/graphblas/core/ss/dtypes.py @@ -104,4 +104,6 @@ def register_new(name, jit_c_definition, *, np_type=None): # array of bool, say) leaves the name free to register again. The type it # made is never used, so no JIT kernel knows its name. _jit_c_type_defs[name] = jit_c_definition + # Its C name and definition are the user's, which ops lifted to it name it by. + rv._jit_c_info = (name, jit_c_definition) return rv diff --git a/graphblas/core/ss/indexbinary.py b/graphblas/core/ss/indexbinary.py index 5aeed9f11..8fd53a27e 100644 --- a/graphblas/core/ss/indexbinary.py +++ b/graphblas/core/ss/indexbinary.py @@ -67,6 +67,8 @@ def register_new(name, jit_c_definition, x_type, y_type, theta_type, ret_type): ``"void my_idxbin (double *z, double *x, GrB_Index ix, GrB_Index jx, "`` ``"double *y, GrB_Index iy, GrB_Index jy, double *theta) "`` ``"{ (*z) = (*x) + (*y) + (*theta) ; }"`` + A UDT made from a numpy dtype is named in C by its ``jit_c_name`` (usually ``gbudt_`` + and its name), not its Python name. x_type : dtype The dtype of the first operand (x). y_type : dtype diff --git a/graphblas/core/ss/indexunary.py b/graphblas/core/ss/indexunary.py index a3a462f59..77eafe067 100644 --- a/graphblas/core/ss/indexunary.py +++ b/graphblas/core/ss/indexunary.py @@ -60,6 +60,8 @@ def register_new(name, jit_c_definition, input_type, thunk_type, ret_type): The C definition as a string of the user-defined function. For example: ``"void diffy (double *z, double *x, GrB_Index i, GrB_Index j, double *y) "`` ``"{ (*z) = (i + j) * fabs ((*x) - (*y)) ; }"`` + A UDT made from a numpy dtype is named in C by its ``jit_c_name`` (usually ``gbudt_`` + and its name), not its Python name. input_type : dtype The dtype of the operand of the indexunary operator. thunk_type : dtype diff --git a/graphblas/core/ss/select.py b/graphblas/core/ss/select.py index 48d16b740..323af9f40 100644 --- a/graphblas/core/ss/select.py +++ b/graphblas/core/ss/select.py @@ -57,6 +57,8 @@ def register_new(name, jit_c_definition, input_type, thunk_type): The C definition as a string of the user-defined function. For example: ``"void woot (bool *z, const int32_t *x, GrB_Index i, GrB_Index j, int32_t *y) "`` ``"{ (*z) = ((*x) + i + j == (*y)) ; }"`` + A UDT made from a numpy dtype is named in C by its ``jit_c_name`` (usually ``gbudt_`` + and its name), not its Python name. input_type : dtype The dtype of the operand of the select operator. thunk_type : dtype diff --git a/graphblas/core/ss/unary.py b/graphblas/core/ss/unary.py index cee7a16a4..695dec5c1 100644 --- a/graphblas/core/ss/unary.py +++ b/graphblas/core/ss/unary.py @@ -55,6 +55,8 @@ def register_new(name, jit_c_definition, input_type, ret_type): jit_c_definition : str The C definition as a string of the user-defined function. For example: ``"void square (float *z, float *x) { (*z) = (*x) * (*x) ; } ;"`` + A UDT made from a numpy dtype is named in C by its ``jit_c_name`` (usually ``gbudt_`` + and its name), not its Python name. input_type : dtype The dtype of the operand of the unary operator. ret_type : dtype diff --git a/graphblas/tests/test_dtype.py b/graphblas/tests/test_dtype.py index e2df81017..a7edc6d7b 100644 --- a/graphblas/tests/test_dtype.py +++ b/graphblas/tests/test_dtype.py @@ -565,13 +565,13 @@ def test_udt_jit_c_info_pinned_at_first_register(): # stable. record_dtype = np.dtype([("pin_a", np.int64), ("pin_b", np.int64)], align=True) udt = dtypes.register_anonymous(record_dtype, "_PinUDT_first") - assert udt.jit_c_name == "_PinUDT_first" + assert udt.jit_c_name == "gbudt__PinUDT_first" udt2 = dtypes.register_anonymous(record_dtype, "_PinUDT_renamed") assert udt2 is udt assert udt.name == "_PinUDT_renamed" # SS-side name is frozen at first register; introspection must reflect that. - assert udt.jit_c_name == "_PinUDT_first" + assert udt.jit_c_name == "gbudt__PinUDT_first" assert "_PinUDT_first" in udt.jit_c_definition @@ -587,12 +587,14 @@ def test_udt_layouts_under_one_name_get_their_own_jit_c_names(): """ small = dtypes.register_anonymous(np.dtype((np.int16, (1, 3, 1))), "_SameJitName") big = dtypes.register_anonymous(np.dtype((np.int16, (2, 1, 7))), "_SameJitName") - assert small.jit_c_name == "_SameJitName" + assert small.jit_c_name == "gbudt__SameJitName" assert big.jit_c_name != small.jit_c_name assert big.jit_c_name in big.jit_c_definition # Nor may a typedef registered in C take a name a UDT already has. with pytest.raises(ValueError, match="unavailable"): - dtypes.ss.register_new("_SameJitName", "typedef struct { int8_t q ; } _SameJitName ;") + dtypes.ss.register_new( + small.jit_c_name, f"typedef struct {{ int8_t q ; }} {small.jit_c_name} ;" + ) # The rest compiles kernels for both layouts, which needs a C compiler the JIT can use. # Asked under 'on': a 'run' the session set (GRAPHBLAS_JIT_C_CONTROL) would answer no. from graphblas.core.ss.jit_config import _enable_jit_for_udt diff --git a/graphblas/tests/test_op.py b/graphblas/tests/test_op.py index abb2bfd48..64bc1880c 100644 --- a/graphblas/tests/test_op.py +++ b/graphblas/tests/test_op.py @@ -2387,7 +2387,7 @@ def test_udt_jit_op_definitions(): typed = getattr(binary, op_name)[udt] lib.GrB_BinaryOp_get_String(typed.gb_obj, buf, lib.GxB_JIT_C_DEFINITION) defn = ffi.string(buf).decode() - assert f"{op_name}_JitOpTest" in defn + assert f"{op_name}_gbudt_JitOpTest" in defn assert "jp" in defn assert "jq" in defn assert expected_c_op in defn @@ -2396,7 +2396,7 @@ def test_udt_jit_op_definitions(): typed = unary.ainv[udt] lib.GrB_UnaryOp_get_String(typed.gb_obj, buf, lib.GxB_JIT_C_DEFINITION) defn = ffi.string(buf).decode() - assert "ainv_JitOpTest" in defn + assert "ainv_gbudt_JitOpTest" in defn assert "jp" in defn # Array UDT JIT definitions, of a length no other test uses (see test_udt_jit_typedef) @@ -2405,14 +2405,14 @@ def test_udt_jit_op_definitions(): typed = binary.plus[arr_udt] lib.GrB_BinaryOp_get_String(typed.gb_obj, buf, lib.GxB_JIT_C_DEFINITION) defn = ffi.string(buf).decode() - assert "plus_Vec37Jit" in defn + assert "plus_gbudt_Vec37Jit" in defn assert "z->v[i] = (x->v[i]) + (y->v[i])" in defn assert "i < 37" in defn typed = unary.ainv[arr_udt] lib.GrB_UnaryOp_get_String(typed.gb_obj, buf, lib.GxB_JIT_C_DEFINITION) defn = ffi.string(buf).decode() - assert "ainv_Vec37Jit" in defn + assert "ainv_gbudt_Vec37Jit" in defn @pytest.mark.skipif("not supports_udfs") diff --git a/graphblas/tests/test_ssjit.py b/graphblas/tests/test_ssjit.py index 3721eee1f..573d0f236 100644 --- a/graphblas/tests/test_ssjit.py +++ b/graphblas/tests/test_ssjit.py @@ -1327,7 +1327,7 @@ def test_udt_jit_c_source_introspection(): udt = dtypes.register_anonymous(record_dtype, "_IntrospectUDT") # dtype-level introspection - assert udt.jit_c_name == "_IntrospectUDT" + assert udt.jit_c_name == "gbudt__IntrospectUDT" assert "typedef struct" in udt.jit_c_definition assert "int64_t introsp_a" in udt.jit_c_definition assert "double introsp_b" in udt.jit_c_definition @@ -1338,7 +1338,7 @@ def test_udt_jit_c_source_introspection(): # Auto-lifted record UDT binary op plus_udt = binary.plus[udt] - assert plus_udt.jit_c_name == "plus__IntrospectUDT" + assert plus_udt.jit_c_name == "plus_gbudt__IntrospectUDT" src = plus_udt.jit_c_source assert src is not None assert "z->introsp_a = (x->introsp_a) + (y->introsp_a)" in src @@ -1346,7 +1346,7 @@ def test_udt_jit_c_source_introspection(): # Auto-lifted record UDT unary op ainv_udt = unary.ainv[udt] - assert ainv_udt.jit_c_name == "ainv__IntrospectUDT" + assert ainv_udt.jit_c_name == "ainv_gbudt__IntrospectUDT" assert "z->introsp_a = -(x->introsp_a)" in ainv_udt.jit_c_source # Array UDT auto-lifted op. Use a shape no other test uses: ``np.dtype`` @@ -1392,12 +1392,12 @@ def test_op_jit_signature_uses_pinned_type_name(): udt2 = dtypes.register_anonymous(record_dtype, "_PinnedRenamed") assert udt2 is udt assert udt.name == "_PinnedRenamed" - assert udt.jit_c_name == "_PinnedOrig" + assert udt.jit_c_name == "gbudt__PinnedOrig" op = binary.plus[udt] # Op signature must reference the pinned struct name, not the renamed one. - assert op.jit_c_name == "plus__PinnedOrig" - assert "_PinnedOrig *" in op.jit_c_source + assert op.jit_c_name == "plus_gbudt__PinnedOrig" + assert "gbudt__PinnedOrig *" in op.jit_c_source assert "_PinnedRenamed" not in op.jit_c_source @@ -2349,8 +2349,9 @@ def test_udt_scalar_broadcast_jit_routing(): assert binary.eq[record, record].jit_c_source is not None # Mixed arithmetic does, with the scalar read once for every leaf assert binary.plus[record, dtypes.INT64].jit_c_source == ( - "void plus_11__BcastJitUV_int64_t (_BcastJitUV *z, const _BcastJitUV *x, " - "const int64_t *y) { z->bcast_u = (x->bcast_u) + (((double) ((*y)))) ; " + "void plus_17_gbudt__BcastJitUV_int64_t (gbudt__BcastJitUV *z, " + "const gbudt__BcastJitUV *x, const int64_t *y) { " + "z->bcast_u = (x->bcast_u) + (((double) ((*y)))) ; " "z->bcast_v = (x->bcast_v) + (((double) ((*y)))) ; }" ) assert binary.plus[dtypes.INT64, record].jit_c_source is not None @@ -2619,7 +2620,13 @@ def test_udt_jit_c_name_too_long_keeps_the_cfunc(): pytest.skip("requires SuiteSparse:GraphBLAS >= 9") _require_jit_on() rec = np.dtype([("tl_x", np.float64), ("tl_y", np.int64)], align=True) - t = dtypes.register_new("TooLongName" + "t" * 120, rec) + # A numpy UDT's C name leaves room for its ops' names, but one registered from C + # source keeps the name given: 125 characters fits, plus_ or eq_ and that does not. + name = "TooLongName" + "t" * 114 + t = dtypes.ss.register_new( + name, f"typedef struct {{ double tl_x ; int64_t tl_y ; }} {name} ;", np_type=rec + ) + assert len(t.jit_c_name) == 125 n = 40 vals = np.zeros(n, t.np_type) vals["tl_x"] = np.arange(n) / 4 @@ -2636,3 +2643,129 @@ def test_udt_jit_c_name_too_long_keeps_the_cfunc(): np.testing.assert_array_equal(got["tl_x"], np.arange(n) / 2) np.testing.assert_array_equal(got["tl_y"], 2 * np.arange(n)) assert same.reduce(monoid.land).new() + + +@pytest.mark.skipif("not supports_udfs") +@pytest.mark.skipif("not _has_jit_set") +def test_udt_field_named_like_a_standard_macro_skips_the_jit(): + """A field named like a macro of the C standard headers the JIT includes skips the JIT. + + The preprocessor expands it inside the struct, so the kernel did not compile: + SuiteSparse raised ``JitError`` (with no message) for that op and set + ``jit_c_control`` to 'load' for the rest of the process. Asked of the C name + only, so nothing compiles here. + """ + from graphblas.exceptions import NoJITWarning + + fields = ["INT_MAX", "UINT8_MAX", "SIZE_MAX", "BUFSIZ", "EXIT_FAILURE", "PRId64", "EDOM"] + # and GraphBLAS's own macros, as GrB_INDEX_MAX, failed the same way + fields += ["SIGINT", "SIG_DFL", "GrB_INDEX_MAX", "GxB_IMPLEMENTATION_MAJOR", "GB_ANY"] + for i, field in enumerate(fields): + record = np.dtype([(field, np.float64), (f"macro_other_{i}", np.float64)], align=True) + udt = dtypes.register_anonymous(record) + assert udt.jit_c_name is None, field + v = Vector.from_coo([0], np.array([(1.0, 2.0)], dtype=record), dtype=udt, size=2) + with pytest.warns(NoJITWarning, match="without JIT"): + result = v.ewise_mult(v, binary.plus).new() + assert result[0].new().value.tolist() == (2.0, 4.0), field + + +@pytest.mark.skipif("not supports_udfs") +@pytest.mark.skipif("not _has_jit_set") +def test_udt_named_like_an_identifier_in_the_kernel_takes_the_jit(): + """A UDT named like an identifier in a JIT kernel compiles, under its ``gbudt_`` C name. + + SuiteSparse's kernels name the type beside their own identifiers. A UDT named like a + parameter of our op definitions (``z``), a local of its templates (``A``, ``p``, ``i``, + ``result``) or something its headers declare (``div``) failed to compile: an empty + ``JitError``, and 'load' for the rest of the process. + """ + _require_jit_on() + for name in ["z", "A", "p", "i", "div", "result"]: + record = np.dtype([(f"kn_{name}_a", np.float64), (f"kn_{name}_b", np.int64)], align=True) + udt = dtypes.register_anonymous(record, name) + assert udt.jit_c_name == f"gbudt_{name}" + vals = np.array([(1.0, 2), (3.0, 4)], dtype=record) + v = Vector.from_coo([0, 1], vals, dtype=udt) + A = gb.Matrix.from_coo([0, 1], [1, 0], vals, dtype=udt) + with _jit_mode("on"): + plus = v.ewise_mult(v, binary.plus).new() + biggest = v.reduce(monoid.max).new() + product = A.mxv(v, semiring.plus_times).new() + assert gb.ss.config["jit_c_control"] == "on", name + assert plus.to_coo()[1].tolist() == [(2.0, 4), (6.0, 8)], name + assert biggest.value.tolist() == (3.0, 4), name + assert product.to_coo()[1].tolist() == [(3.0, 8), (3.0, 8)], name + + +@pytest.mark.skipif("not supports_udfs") +@pytest.mark.skipif("not _has_jit_set") +def test_udt_long_name_gets_a_c_name_that_fits(): + """A UDT whose ``gbudt_`` C name would not leave room for its ops' names gets a synthetic one. + + SuiteSparse refuses a name of 128 characters or more. A type with such a name was kept + with an empty one, so its first kernel, a transpose say, raised ``JitError``; and with a + C name of 119 to 127 characters, ``floordiv_`` or ``plus_`` and it did not fit, so the + op lost its JIT with a NoJITWarning that blamed the fields. + """ + from graphblas.core.operator.udt_codegen import _max_type_c_name_len + + _require_jit_on() + assert _max_type_c_name_len() == 118 # 127 less floordiv_ + for length in [112, 113, 119, 200]: + record = np.dtype([(f"fit{length}_a", np.float64), (f"fit{length}_b", np.int64)]) + udt = dtypes.register_anonymous(record, "n" * length) + if length == 112: + assert udt.jit_c_name == "gbudt_" + "n" * 112 # 118 characters + else: + assert udt.jit_c_name.startswith("_gbudt_"), length + for op in [binary.floordiv[udt], binary.plus[udt], binary.eq[udt], unary.ainv[udt]]: + assert op.jit_c_source is not None, (length, op) + vals = np.array([(1.0, 2), (3.0, 4)], dtype=record) + A = gb.Matrix.from_coo([0, 1], [1, 0], vals, dtype=udt) + with _jit_mode("on"): + B = A.T.new() + C = A.ewise_mult(A, binary.plus).new() + assert B.to_coo()[2].tolist() == [(3.0, 4), (1.0, 2)], length + assert C.to_coo()[2].tolist() == [(2.0, 4), (6.0, 8)], length + + +def test_udt_c_name_keeps_python_graphblas_own_names(): + # A type python-graphblas makes itself has a name starting with _gb, which no kernel + # identifier starts with either, so it keeps that name as its C name; a user's name gets + # the gbudt_ prefix, even one that starts with it. + from graphblas.core.operator.udt_codegen import _pick_c_type_name + + assert _pick_c_type_name("Edge") == "gbudt_Edge" + assert _pick_c_type_name("gbudt_Edge") == "gbudt_gbudt_Edge" + for name in ["_gbudt_7", "_gbnest_3", "_gbinternal_fp64x16"]: + assert _pick_c_type_name(name) == name + + +@pytest.mark.skipif("not supports_udfs") +@pytest.mark.skipif("not _has_jit_set") +def test_udt_from_c_source_lifts_ops_under_its_own_c_name(): + """An op lifted to a type registered from C source names the type as its C source does. + + The prefix that keeps a numpy UDT's C name clear of kernel identifiers was also put on + a C-source type's name, so ``binary.plus[t]`` defined ``plus_gbudt_cpt_t`` on a type + ``gbudt_cpt_t`` that SuiteSparse does not know: ``JitError``, then 'load'. + """ + _require_jit_on() + np_type = np.dtype([("cs_x", np.float64), ("cs_y", np.float64)], align=True) + t = dtypes.ss.register_new( + "cs_lift_t", "typedef struct { double cs_x ; double cs_y ; } cs_lift_t ;", np_type=np_type + ) + assert t.jit_c_name == "cs_lift_t" + assert binary.plus[t].jit_c_name == "plus_cs_lift_t" + assert binary.eq[t].jit_c_name == "eq_cs_lift_t" + v = Vector.from_coo([0, 1], np.array([(1.0, 2.0), (3.0, 4.0)], dtype=np_type), dtype=t) + with _jit_mode("on"): + r = v.ewise_mult(v, binary.plus).new() + same = (v == v).new() + assert gb.ss.config["jit_c_control"] == "on" + assert r.to_coo()[1].tolist() == [(2.0, 4.0), (6.0, 8.0)] + assert same.to_coo()[1].tolist() == [True, True] + # A numpy UDT's own name always gets the prefix, even one that starts with it + udt = dtypes.register_anonymous(np.dtype([("cs_pre", np.float64)]), "gbudt_CsPre") + assert udt.jit_c_name == "gbudt_gbudt_CsPre"