Skip to content

Drop unify's ignored scalar flags inside the library, and share the UDT leaf walkers - #665

Open
eriknw wants to merge 7 commits into
56-udt-jit-edge-casesfrom
57-udt-cleanups
Open

eriknw wants to merge 7 commits into
56-udt-jit-edge-casesfrom
57-udt-cleanups

Conversation

@eriknw

@eriknw eriknw commented Nov 4, 2026

Copy link
Copy Markdown
Member

Stacked on #663. Follow-ups to #591 from review. This is a refactor with no change in behavior. Every existing
test passes unchanged, and one test is added.

unify's scalar flags. dtypes.unify has ignored is_left_scalar and is_right_scalar
since #591, which types a literal before it reaches unify. The library no longer passes
them:

  • the fill-value and ewise_union calls in Vector, Matrix and Scalar;
  • get_typed_op (in graphblas.core), which took them only to pass them on and no longer
    takes them.

unify keeps both keywords, and its docstring now says it accepts and ignores them.

Walkers.

  • _has_half_leaf, _has_narrow_float_leaf, _has_duration_leaf,
    _numba_untyped_array_element and _raw_bytes_element now test the leaves of one walker,
    dtypes._iter_leaves. It walks records, nested records, arrays of records and arrays of
    arrays, as each of them did.
  • _check_leaves_compare uses _leaf_dtypes, which its inner function repeated.
  • _int_beyond_leaf is gone. _beyond_as_infinity read a literal by the same rules. It now
    returns None when no int is beyond its leaf's range, so a comparison walks the literal
    once.
  • _check_udt_pair and _corresponding_leaves ask _record_leaf_mismatch whether two records
    pair: the same nesting, the same names in the same order, and the same shape for each field.
    Each words its reason as before. Array UDTs keep their own rules, which differ on purpose:
    operands broadcast, but a stored element does not.

Some walkers stay separate. Each reads a literal by its own rules, and merging them would
change a result:

  • _refuse_misfit_fields: a non-sequence has no parts.
  • _weak_literal_udt's record reader: only a number fills a record, and a wrong length
    refuses the whole literal.
  • _nats_in_leaf_units: a list is a sequence of elements.
  • _as_record_parts: it turns a dict into a tuple, and keeps a scalar as it is.
  • _leaf_dtypes: it treats an array of records as one leaf.

Tests. A new row in test_jit_udt_np_type_numba_cannot_type checks a field that is an
array of arrays of bool, which numpy keeps unflattened. It still gets the message for an array
Numba cannot type.

A differential check compared the old and new walkers on 362 dtypes:

  • every scalar kind, including text, bytes, raw void, dates, durations, float16 and complex;
  • arrays of them, and arrays of arrays;
  • random records with array fields;
  • nested records and arrays of records;
  • titled fields and out-of-order offsets.

It also covered 8,014 literal values and 131,044 dtype pairs. All 665,406 results matched on
numpy 1.24.4, 2.1.3, 2.4.6 and 2.5.4. The full suite passes with --runslow on both backends,
without Numba, and with the CI's C-scalar substitution.

Questions for Erik

# Question Recommendation
1 Deprecate unify's is_left_scalar and is_right_scalar (a DeprecationWarning when given), or keep accepting them silently? Keep them silent here. A deprecation can go with the next release that removes other things.
2 get_typed_op no longer takes the two keywords. It is in graphblas.core, and nothing in the stack, graphblas-algorithms or nx-graphblas passes them. Accept them again for other callers? No, graphblas.core is private.
3 _record_leaf_mismatch is the one walker change that adds lines (+21), to put the record pairing rule in one place. Keep it? Keep it. The rule is in one place, and its docstring shows where the rules for an op and for a cast part. It is the last commit, so it can be dropped alone.

🤖 Generated with Claude Code

@eriknw
eriknw added this pull request to stack #666 November 4, 2026 20:51
@codecov

codecov Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.82353% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 99.28%. Comparing base (d00f5f6) to head (8c960f6).

Additional details and impacted files
@@                    Coverage Diff                    @@
##           56-udt-jit-edge-cases     #665      +/-   ##
=========================================================
+ Coverage                  98.66%   99.28%   +0.62%     
=========================================================
  Files                        109      109              
  Lines                      35869    35852      -17     
  Branches                    4100     4087      -13     
=========================================================
+ Hits                       35390    35597     +207     
+ Misses                       254      144     -110     
+ Partials                     225      111     -114     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@eriknw
eriknw force-pushed the 57-udt-cleanups branch 3 times, most recently from f666203 to 080a941 Compare November 9, 2026 15:51
@eriknw
eriknw force-pushed the 57-udt-cleanups branch 2 times, most recently from 5ec2d02 to 427a2fa Compare November 10, 2026 17:12
@eriknw
eriknw force-pushed the 57-udt-cleanups branch 14 times, most recently from b9fbfa3 to 8387a9e Compare November 11, 2026 05:42
@eriknw
eriknw removed this pull request from stack #666 November 12, 2026 06:13
@eriknw
eriknw changed the base branch from 56-udt-jit-edge-cases to main November 12, 2026 06:15
@eriknw
eriknw changed the base branch from main to 56-udt-jit-edge-cases November 12, 2026 06:16
@eriknw
eriknw added this pull request to stack #670 November 12, 2026 06:17
@eriknw
eriknw force-pushed the 57-udt-cleanups branch 10 times, most recently from 5dc0cba to 957e2be Compare November 13, 2026 04:37
unify has ignored is_left_scalar and is_right_scalar since a literal is typed
before it gets there, so the fill-value and ewise_union calls in Vector,
Matrix and Scalar stop passing them.
get_typed_op took is_left_scalar and is_right_scalar only to hand them to
unify, which ignores them. Drop them from get_typed_op and from every
caller in Vector, Matrix, Scalar and _get_typed_op_from_exprs.
_has_half_leaf, _has_narrow_float_leaf, _has_duration_leaf,
_numba_untyped_array_element and _raw_bytes_element each walked records,
nested records, arrays of records and arrays of arrays to test each leaf.
They now test the leaves dtypes._iter_leaves yields, which walks them the
same way. A test checks that a field holding an array of arrays of bool,
which numpy keeps unflattened, still gets the message for an array Numba
cannot type.
The inner leaves() of _check_leaves_compare was _leaf_dtypes again.
_int_beyond_leaf and _beyond_as_infinity read a literal's parts by the
same rules, and the comparison path called the first only to decide
whether to call the second. _beyond_as_infinity now returns None when
there is no such int, and _int_beyond_leaf goes.
_check_udt_pair and _corresponding_leaves each compared the leaf paths and
then the field shapes of two records, one to raise KeyError and one to
return a reason. _record_leaf_mismatch now says where they differ, and
each caller words it as before. The rules for array UDTs stay apart:
operands broadcast, a stored element does not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant