How to drop zeros for a matrix ? #539

Activity

1. eriknw commented on Feb 20, 2024

   

   eriknw

   on Feb 20, 2024

   Member

   More actions

   Thanks for the great questions! My answer is going to be a small tour of python-graphblas. It'll be kind of a lot, and I'll be happy to clarify anything if you have any questions.

   To the first question about using assign as is done in LAGraph--you need to use a value mask (the second argument in GrB_assign) and use the "replace" descriptor. In python-graphblas, this can be done as:

   A(A.V, replace=True) << A

   where A.V indicates "value mask" (instead of A.S, which indicates "structural mask"), and replace=True sets the "replace" descriptor.

   It's also fine to add brackets to make the assign more explicit, such as:

   A[:, :](A.V, replace=True) << A
   A(A.V, gb.replace)[:, :] << A

   I need to count non-zero items columnwise. But gb.binary.gt(mxIn , 0) is BOOL and cannot do sum

   Sum reductions on boolean values are possible if you're explicit with the typecasting, such as:

   gb.binary.gt(A, 0).reduce_columnwise(gb.monoid.plus[int])

   For now, we chose not to treat boolean dtypes as "little integers".

   SuiteSparse:GraphBLAS made comparator functions that begin with is that return the same type as the input, so doing e.g. isgt(A, 0) can return integer instead of boolean, and sum reduction "just works":

   gb.binary.isgt(A, 0).reduce_columnwise()

   I think there are two other good ways of doing the operation you're trying to do. The first is to use to use "select" to drop the zeros:

   gb.select.valuegt(A, 0).reduce_columnwise()  # or use select.valuene

   Another way is to use aggregators that are unique to python-graphblas:

   A.reduce_columnwise(gb.agg.count_nonzero)

   I think this is nice, because it's one statement and works for arbitrary non-zero values (i.e., the values don't need to be 0 and 1). This particular aggregator works by creating an iso-valued vector with the value 0, then doing a vector-matrix multiply with the plus_isne semiring.

   gb.unary.one(mxIn).new() just set every element to one including zeros and false(s).

   I think I see what you want to do here. You can pass a "value mask" to .new like this to drop the zeros:

   B = gb.unary.one(A).new(mask=A.V)
   B.reduce_columnwise()

   Just to be thorough (I probably wouldn't recommend this over the other ways), you can also cast the matrix to bool dtype:

   A.dup(bool).reduce_columnwise("+[int]")

   Whew!

   There we go. Most or all of these ways should be pretty fast. Heh, I hope I answered more questions then I raised. I'll be happy to answer more.

   🚀React with 🚀1Reacted by Sultan Orazbayev
2. eriknw commented on Feb 20, 2024

   

   eriknw

   on Feb 20, 2024

   Member

   More actions

   And to answer the issue title:

   How to drop zeros for a matrix ?

   Using "select" is preferred.

   In-place:

   A(A.V, replace=True) << A
   # or
   A << gb.select.valuene(A, 0)
   # or
   A << A.select("!=", 0)

   Create new objects:

   B = gb.select.valuene(A, 0).new()
   # or
   B = A.select("!=", 0).new()
   # or
   B = gb.unary.identity(A).new(mask=A.V)
   # or
   B = A[:, :].new(mask=A.V)
3. galaxy001 commented on Feb 20, 2024

   

   galaxy001

   on Feb 20, 2024

   Last edited by galaxy001

   Author

   More actions

   What is the difference between B << gb.select.valuene(A, 0) and B << gb.select.valuene(A, 0).new() ? There is doc for dup() but no doc for new().
   I guess new() will cause the calculation of selection been run. Maybe sth. like lazy calculation.

   So, will gb.select.valuene(A, 0).reduce_columnwise() and gb.select.valuene(A, 0).new().reduce_columnwise() be the same and takes same CPU time and same memory size ?

   If I need to use a lazy calculation var more than once, do I need to append new() since the calculation will be run each time I use that var without tailing new() ?

   ---

   Thanks for your tour. I find dropping zeros first is better since reduce_columnwise on a column contains only zeros and NULLs will produce zero. I prefer to also get rid of zeros in results.