Repository navigation
Conversation
eriknw
marked this pull request as ready for review
August 4, 2026 16:07
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 4, 2026 16:12
adc87bf to
d4117f9
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
4 times, most recently
from
August 5, 2026 18:03
a4829ec to
5380d4f
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 5, 2026 18:05
5380d4f to
90cb7d9
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 6, 2026 07:59
90cb7d9 to
7ac6414
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 6, 2026 15:39
7ac6414 to
44f7d06
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 6, 2026 15:41
44f7d06 to
89b2cf8
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 6, 2026 20:36
89b2cf8 to
89235b6
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 6, 2026 20:42
89235b6 to
397d63f
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 7, 2026 02:48
397d63f to
f7241d9
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 7, 2026 05:09
f7241d9 to
7611a08
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
August 26, 2026 17:30
7611a08 to
6316a13
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
September 18, 2026 03:09
6316a13 to
58d4c38
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
2 times, most recently
from
October 3, 2026 04:57
bb6a890 to
e1a2cc3
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
October 4, 2026 06:21
e1a2cc3 to
34c2ce0
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
October 8, 2026 21:40
34c2ce0 to
53aec58
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
October 9, 2026 03:06
53aec58 to
5add7a7
Compare
eriknw
force-pushed
the
24-operator-resolution
branch
from
October 9, 2026 05:25
5add7a7 to
6a691db
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 49-masked-literal-assign #604 +/- ##
============================================================
- Coverage 99.33% 99.02% -0.32%
============================================================
Files 106 106
Lines 33052 33188 +136
Branches 3841 3863 +22
============================================================
+ Hits 32833 32863 +30
- Misses 147 216 +69
- Partials 72 109 +37 🚀 New features to boost your workflow:
|
get_typed_op imported the agg module inside the function on every operator resolution, to isinstance-check Aggregator and TypedAggregator. The import had to be function-local because agg.py imports get_typed_op, so a module-level import would be a cycle. agg.py now registers its two classes with operator/utils.py when it loads, and get_typed_op consults the registry instead of running an import statement per call. The None guard is safe because those classes are the only way to make an Aggregator: before agg.py finishes loading no instance can exist, and agg.py's own body never hands one to get_typed_op (its two call sites pass a monoid and a semiring, both from inside functions rather than at module level). This does not change when the agg module loads. core/operator/__init__.py imports Aggregator eagerly, so agg is already in sys.modules before any user code can reach get_typed_op; the function-local import was repeating a sys.modules lookup rather than deferring a module load.
…ons (gh-239) gh-239 asks for value, row, column and index helpers in the indexunary namespace, mirroring the four that select already has. This adds three of them: indexunary.value(A > 3) returns the same expression as A.apply(indexunary.valuegt, 3), with the same input validation, the same lt-to-le and ge-to-gt thunk shifts, and the same errors as select's helpers, adapted to name indexunary. The shared resolution logic moves to core/operator/utils.py; select delegates to it and is unchanged. index is deliberately not added, so this closes three quarters of gh-239. indexunary.index already exists as an operator alias for rowindex, and is used as one (v.apply(indexunary.index) and indexunary.index(v) are both tested), so a helper function of that name would shadow a working operator. Adding it needs a decision about that alias, which is a maintainer call. Vector index comparisons are meanwhile reachable through indexunary.row(v < k) or the indexle and indexgt aliases. New matrix and vector tests compare each helper against the explicit apply, and cover the BOOL result dtype, the thunk shifts, the scalar expression path, and the error cases.
All eight operator-namespace from_string helpers (unary, binary, monoid, semiring, select, indexunary, agg, op) had no docstring at all. Add numpydoc docstrings describing the accepted string forms: a namespace name, a dotted path such as numpy.mod, shorthand symbols such as + or >=, and the [dtype] typing suffix. Each carries runnable examples. test_from_string gains select and indexunary cases and asserts every namespace's from_string is now documented. Also fix a stale example in the indexunary from_string error message. It offered 'row_index', which does not resolve; the operator is 'rowindex'.
agg.py registers the Aggregator classes when core.operator loads it, at import, and no call reaches this check before then, so the arm where _Aggregator is still None never runs (the guard is there in case one ever does).
…by name
_from_string took any attribute of the namespace module, so the new
helper functions, and private names and methods, came back as operators:
indexunary.from_string("value") (also "row", "column", "VALUE")
before: <function value> ("value[int]": TypeError, not subscriptable)
after: ValueError: Unknown indexunary string: 'value'
as on the parent, and as the from_string docstrings say (IndexUnaryOp).
select.from_string("value") and op.from_string("register_new") (a bound
method) are refused too. Applying a helper reached Numba as a UDF:
A.apply(indexunary.value)
before: UdfParseError: unary.value does not work with (INT64): Untyped
global name '_resolve_expr': ...
after: TypeError: indexunary.value is a helper, not an operator: it
takes a comparison and returns its expression, as in
indexunary.value(A > 3). To apply an op, give one such as
indexunary.valuegt.
_resolve_index_expr took any expression, so a wrong one failed somewhere
inside, or reached the op it picked, and a Scalar's comparison crashed in
the < and >= shift, which read the thunk before taking it out of its Vector:
select.row(s < 1), indexunary.row(s >= 0.5) (s a Scalar)
before: AttributeError: 'Vector' object has no attribute 'value'
after: s, and False
indexunary.value(abs(A)) before: IndexError: list index out of range
indexunary.value(A.mxm(A)) before: AttributeError: 'Matrix' object ...
indexunary.value(A.apply(gt, left=3)) before: TypeError: Do not pass `left` ...
indexunary.value(A.apply(plus, 3)) before: ValueError: Unknown or unregistered
indexunary method: valueplus
after: TypeError: indexunary.value takes a comparison of a Vector, Matrix or
Scalar with a scalar on its right, as in indexunary.value(x <= 5);
got M_0.apply(unary.abs[INT64])
indexunary.row(A == 3)
before: ValueError: Unknown or unregistered indexunary method: roweq
after: ValueError: indexunary.row compares indices with <, <=, > or >=, not ==
indexunary.row(A < Scalar(float)) (an empty thunk)
before: TypeError: unsupported operand type(s) for -=: 'NoneType' and 'int'
after: EmptyObject, as A.apply(indexunary.rowle, Scalar(float)) raises
_check_index_expr accepts a built-in comparison applied with a scalar on
the right (a Scalar's is an ewise_mult of two Vectors), and for row and
column only the four orderings. The shift now runs after the Scalar unwrap
and leaves an empty thunk to the op. With these checks the "Unknown or
unregistered" raise is unreachable again, so its pragma stands. Select's
helpers share the function and change the same way; main's select had
the same crashes.
…ss does
_resolve_index_expr looked the op up in the namespace's globals, which
hold an op registered with lazy=True only once something touches it:
indexunary.register_new("rowlt", lambda x, i, j, t: i < t, lazy=True)
indexunary.row(A < 2.5) (rows 0 to 3)
before: [True, True, False, False] (the <= shift), then [True, True,
True, False] once indexunary.rowlt was touched
after: [True, True, True, False] both times, from the op
_namespace_op builds such an op through the module's __getattr__, as
indexunary.rowlt would. select.row does the same; main's select did too.
The comment said no Aggregator can exist before agg.py registers its classes, but agg.py builds every built-in Aggregator at module level first. What holds is that none reaches get_typed_op until then.
"Add docstrings to the operator from_string helpers (GH #513)" fixed the example in that message (row_index, which does not resolve, became rowindex) without a test; on its parent this one fails.
select.index(v < 4) filters by index, but indexunary.index is the rowindex op (a helper would shadow it), so indexunary.index(v < 4) gives each entry's index without an error. The note for that was a code comment; the row helper's docstring now says to use row for a Vector.
Port of PR A's "Compare indices with the bound given in select.row, column and index" to _resolve_index_expr, which the select and indexunary helpers share here. When #604's chain is replayed over A, its "Add indexunary.value/row/column helpers accepting comparison expressions (gh-239)" conflicts in graphblas/select/__init__.py; take that commit's own version of the file (select delegates here), and this commit carries A's fix into the shared function. The helpers read the thunk from the comparison x < k, which types k beside x's values: beside INT8 or UINT8, 300 is infinity, and beside FP32, 2**24 + 1 rounds. indexunary.row(B <= 299) on a 400-row INT8 matrix was True in all 400 rows, as select.row(B < 300) kept them. Now they compare indices with the number the caller gave (the expression's _right_literal, from A), or the thunk Scalar's value, made a whole number: i < t is i <= ceil(t) - 1, i >= t is i > ceil(t) - 1, and i <= t and i > t take floor(t). An op registered for < or >= (a user's rowlt) gets the number as given. An empty Scalar thunk still reaches the op, which raises EmptyObject. Also r604's R1: a Scalar's .apply keeps its thunk a Scalar, so taking it out of a Vector crashed: select.value(s.apply(binary.lt, right=10)) raised AttributeError on suitesparse and TypeError on vanilla. The thunk is now taken out of a Vector only when it is one. Tests: indexunary.row and column on Matrix (INT8, UINT8, FP32, transposed, fractional bounds), Vector and Scalar, and the R1 case.
The same change as PR A's "Take the thunk's value as an index bound that is not a number", in _index_bound here, which the select and indexunary helpers share: indexunary.row(A < np.array(3)) kept rows 0 to 3.
An IndexUnaryOp registered with lazy=True waits in indexunary._delayed, and its SelectOp is made only when it is built. select's helpers looked in select's namespace alone, so select.row(A < 2.5) took the <= shift until something touched indexunary.rowlt, and then the op (r604 R3). _namespace_op now builds the indexunary op for select when select has none by that name, as attribute access on indexunary would. The test of a lazily registered op could not fail since the helpers compare indices with the bound given: its rowlt, i < t, answered as the <= shift does, so it passed with the lazy lookup removed (r604 P1). Its rowlt is now i == 0, which the shift does not give, and select.row runs first. It fails with either the lazy lookup or this change taken out.
… here too The same change as on #661, in _index_bound's twin in core/operator/utils.py, which the select and indexunary helpers share here: A of 5 rows: indexunary.row(A < nan), (A <= nan) before: True for row 0 only indexunary.row(A > nan), (A >= nan) before: True for rows 1 to 4 after: False for every row, and select.row keeps none (main kept row 0, or rows 1 to 4) _index_bound also reads a value only from a Scalar thunk now, as on #661, where that restored the op's TypeError for A.apply(binary.le, left=3). Here _check_index_expr refuses that form first, so the two copies stay alike without a test of it.
#661's test of select.row(A.apply(binary.le, left=3)) expects the op's TypeError, as main gives. Here the helpers refuse such an expression first, with a TypeError that names the helper ("Check the comparison given to the value, row and column helpers"), so the test now expects that message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #648. Operator lookup: a registry for the aggregator check in
get_typed_op,value,rowandcolumnhelpers in the indexunary namespace (gh-239), docstrings forevery
from_string(GH #513), and fixes to the helpers andfrom_stringfrom review.None of it changes which typed op is picked:
get_typed_opgives the same op as beforefor every built-in op, all 13 x 13 built-in dtype pairs, plain and with a scalar on either
side, and the string forms (212,218 calls compared).
Break the per-call agg import in get_typed_op with a type registry.
get_typed_opimported the agg module inside the function to check for Aggregator and TypedAggregator,
for every op it did not resolve earlier (aggregators, strings and callables; built-in
BinaryOp, Monoid and Semiring returned before it). agg.py now registers the two classes
when it loads, and
get_typed_opreads them from there. When agg loads does not change:core.operatorimports it eagerly. Exclude the before-registration arm ofget_typed_op's Aggregator check adds the coverage pragma, and Say why get_typed_op can
skip the Aggregator check before registration corrects the comment: agg.py builds its
Aggregators before it registers the classes, but none reaches
get_typed_opuntil then.Add indexunary.value/row/column helpers accepting comparison expressions (gh-239).
indexunary.value(A > 3)isA.apply(indexunary.valuegt, 3), asselect.value(A > 3)is
A.select(select.valuegt, 3).rowandcolumnrewrite<and>=with a thunkshift, as select's do. The shared code moves to
core/operator/utils.py(
_resolve_index_expr), and select calls it.indexis not added, becauseindexunary.indexis already therowindexop, used as one (v.apply(indexunary.index)).Say in indexunary.row's docstring that indexunary.index is an op tells Vector users to
use
row:indexunary.index(v < 4)gives each entry's index, whileselect.index(v < 4)filters by index. This closes three quarters of gh-239.
Fixes to the helpers, which select's share:
comparison crashed in the
<shift (select.row(s < 1): AttributeError, also on main).A wrong expression failed somewhere inside:
value(abs(A))with IndexError,value(A.mxm(A))with AttributeError, andvalue(A.apply(plus, 3))with "Unknown orunregistered indexunary method: valueplus". Each now raises a TypeError that names the
helper and shows the expression.
row(A == 3)says rows compare only with<,<=,>,>=, and an empty Scalar thunk raises EmptyObject, as the explicit op does.rowltregistered withlazy=Truewas used only after something touched it, sorow(A < 2.5)changed its answer.select's side: its SelectOp is made only when the indexunary op is built, so
select.row(A < 2.5)took the<=shift until something touchedindexunary.rowlt._namespace_opnow builds the indexunary op when select has none by that name. The test ofa lazily registered op could not fail, since its
rowlt(i < t) answered as the shiftdoes; its
rowltis nowi == 0, andselect.rowruns first.helpers are functions in the namespace, so
indexunary.from_string("value")returnedone, and
A.apply(indexunary.value)compiled it as a UDF and failed in Numba. Now thefirst is "Unknown indexunary string", as before this PR and as the docstring says, and
the second is a TypeError saying it is a helper.
op.from_string("register_new")nolonger returns a bound method.
Add docstrings to the operator from_string helpers (GH #513). All eight namespaces'
from_stringhad none; each now has a numpydoc docstring with runnable examples (names,dotted paths such as
numpy.mod, symbols such as+and>=, the[dtype]suffix). Italso fixes the example in the indexunary error message:
'row_index'does not resolve,the op is
'rowindex'. Test the example in indexunary.from_string's error messagecovers that.
Known, fixed below this PR. The row, column and index helpers read their thunk from
the comparison
A < k, which typeskbeside the values. Beside INT8, 300 becomesinfinity, and beside FP32, 2**24 + 1 rounds, so
select.row(B < 300)kept every row of a400-row INT8 matrix. A fractional thunk was truncated toward zero. This came in with #591's
literal typing (main is right) and is fixed in #661 (follow-ups to #591), in select's
helpers. This PR moves that code into
_resolve_index_expr, which select's and indexunary'shelpers share, so "Compare indices with the bound given in the row and column helpers"
carries the fix there, with tests for
indexunary.rowandcolumn, and "Take the thunk'svalue as an index bound that is not a number, here too" carries #661's later fix for a
bound given as a Scalar expression or a 0-d array. "Select no index for a NaN bound in the
row, column and index helpers, here too" carries #661's NaN fix into the shared function, so
indexunary.row(A < nan)is False for every row (it was True for row 0), and keeps the twocopies of
_index_boundalike. The same commit fixesselect.value(s.apply(binary.lt, right=10)), which crashed (on main too)."Expect select.row's own TypeError for a left-operand comparison here" updates #661's test
of
select.row(A.apply(binary.le, left=3)): this PR's input check refuses it before the opdoes, with a TypeError that names the helper.
Tests:
test_indexunary_helpersin test_matrix.py and test_vector.py (each helper againstthe explicit apply, BOOL results, the shifts, the Scalar path, and the error cases),
test_index_helpers_find_a_lazily_registered_op, Scalar row helpers intest_selectintest_scalar.py, and in
test_from_stringthe documented-docstring check, thefrom_stringrefusals and the applied-helper TypeError.Stack created with GitHub Stacks CLI • Give Feedback 💬