From 60ef99fadb6e41c4168139e784b828a99386515e Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Thu, 27 Aug 2026 11:19:40 -0500 Subject: [PATCH 01/19] Pin the CI interpreter and its ABI in the update-env step conda-forge ships two ABIs for CPython 3.13+, and the "Update env" step did not restate ``python``, so the solver was free to replace the interpreter setup-miniconda installed. Three of the last sixteen python 3.14 draws on ubuntu-24.04-arm did exactly that: the log reads ``python 3.14.7-h6bfacdd_100_cp314 --> 3.14.7-hefd156b_0_cp314t`` and pytest then reports "Python: 3.14.7 free-threading build". Two of the three passed only because they also drew pandas 3.0. The third drew pandas 2.3, whose extension raises "The global interpreter lock (GIL) has been enabled to load module 'pandas._libs.pandas_parser'", and ``filterwarnings = error`` turns that into a collection error. It took test_formatting.py and test_recorder.py down with it, the second only because the first had left pandas half-imported. The window opened on 2026-08-19, when conda-forge uploaded python 3.14.7 for linux-aarch64 in both ABIs. Re-solving the failing job's own spec set per platform: linux-aarch64 and osx-arm64 both pick ``cp314t`` today, so macos-latest is on the same fuse and has simply not drawn 3.14 with an unlucky pandas yet. linux-64, osx-64 and win-64 pick the GIL build either way, which is why this has been an arm-only symptom. ``python-gil`` pins ``python_abi``; restating ``python=`` pins the version, since ``python-gil`` alone constrains only the ABI and its own version is left open. setup-miniconda's ``python-version`` is equally ABI-free, so this step is where both get settled. Re-solved with the pair added: every platform lands on a ``cp314`` build, and no draw becomes unsatisfiable or downgrades numpy, numba, pandas or psg, including python 3.11 with numpy 1.24 and scipy 1.9. python-gil has builds for 3.9 through 3.14, so one spec covers every version in the pool. The pypy guard matches the neighbouring lines: ``pypy python-gil`` is unsatisfiable, so an unguarded spec would hard-fail a pypy draw if one is ever added back. python-suitesparse-graphblas pins the same pair in ``.github/workflows/test.yml``, and asks for ``python-freethreading`` on the matrix entries that want the other ABI. An entry like that would be worth having here too, but it is a coverage decision rather than a fix, so it is not in this commit. --- .github/workflows/test_and_build.yml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 8dec76f13..99fbb580f 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -203,6 +203,17 @@ jobs: echo "versions: np${npver} sp${spver} pd${pdver} ak${akver} nx${nxver} numba${numbaver} yaml${yamlver} sparse${sparsever} psg${psgver}" set -x # echo on + # conda-forge ships two ABIs for CPython 3.13+, and this step did not + # restate ``python``, so the solver was free to replace the interpreter + # setup-miniconda installed: a 3.14 draw on ubuntu-24.04-arm became + # 3.14.7-*_cp314t, where pandas 2.3 raises a GIL RuntimeWarning that + # filterwarnings=error turns into a collection error. Restating the + # version pins the interpreter and ``python-gil`` pins python_abi to + # the GIL build; setup-miniconda's own ``python-version`` above is + # equally ABI-free, so this covers that step too. + # python-suitesparse-graphblas pins the same pair in + # .github/workflows/test.yml, and asks for python-freethreading on the + # matrix entries that want the other ABI. $(command -v mamba || command -v conda) install -c nodefaults \ packaging pip pytest coverage pytest-randomly cffi donfig tomli c-compiler make \ pyyaml${yamlver} ${sparse} pandas${pdver} scipy${spver} numpy${npver} ${awkward} \ @@ -212,6 +223,7 @@ jobs: ${{ steps.sourcetype.outputs.selected == 'upstream' && 'cython' || '' }} \ ${{ steps.sourcetype.outputs.selected != 'wheel' && '"graphblas>=7.4"' || '' }} \ ${{ contains(steps.pyver.outputs.selected, 'pypy') && 'pypy' || '' }} \ + ${{ !contains(steps.pyver.outputs.selected, 'pypy') && format('python={0} python-gil', steps.pyver.outputs.selected) || '' }} \ ${{ matrix.os == 'windows-latest' && 'cmake' || 'm4' }} \ # ${{ matrix.os != 'windows-latest' && 'pytest-forked' || '' }} # to investigate crashes - name: Build extension module From 721993cdce60f4a14affc5950a487af447a37347 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:09:04 -0500 Subject: [PATCH 02/19] Bump the GitHub Actions and pin them to commit SHAs Every action moves to its newest release: checkout 7.0.1, setup-python 7.0.0, setup-miniconda 4.1.0, codecov-action 7.1.1, upload-artifact 7.0.2, gh-action-pypi-publish 1.14.2, and action-tmate 3.24 (pre-commit/action stays at 3.0.1, its newest). This supersedes the open Dependabot PRs #572, #573, #575, #577 and #578. Each is pinned to the commit of its release, with the version in a comment, since a tag can be moved to other code. Dependabot updates the pin and the comment together. The SHAs are the same ones python-suitesparse-graphblas and afar pin. Dependabot now waits seven days after a release before proposing it and groups all action bumps into one PR a week, as those two repositories do. None of the major bumps changes an input this repository uses. Most are the move to Node.js 24; setup-python 7 drops the ``pip-install`` input, checkout 7 refuses fork checkouts under pull_request_target and workflow_run, and upload-artifact 7 adds unzipped single-file uploads. One input changes: * setup-miniconda deprecated ``auto-activate-base`` in favor of ``auto-activate`` (already in 3.3), and every run logged a warning for it. --- .github/dependabot.yml | 9 +++++++++ .github/workflows/audit_versions.yml | 8 ++++---- .github/workflows/debug.yml | 4 ++-- .github/workflows/imports.yml | 4 ++-- .github/workflows/lint.yml | 6 +++--- .github/workflows/publish_pypi.yml | 8 ++++---- .github/workflows/test_and_build.yml | 10 ++++++---- 7 files changed, 30 insertions(+), 19 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 5ace4600a..94e8d4b16 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -4,3 +4,12 @@ updates: directory: "/" schedule: interval: "weekly" + # Wait a week after a release before proposing it, so a compromised or + # broken release has time to be noticed and yanked. + cooldown: + default-days: 7 + # One PR per week for all action bumps instead of one PR per action. + groups: + github-actions: + patterns: + - "*" diff --git a/.github/workflows/audit_versions.yml b/.github/workflows/audit_versions.yml index 646e96f59..162713824 100644 --- a/.github/workflows/audit_versions.yml +++ b/.github/workflows/audit_versions.yml @@ -43,10 +43,10 @@ jobs: name: Audit against PyPI runs-on: ubuntu-latest steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: "3.12" - run: python -m pip install --upgrade pip packaging @@ -64,10 +64,10 @@ jobs: run: shell: bash -el {0} steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: conda-incubator/setup-miniconda@v3 + - uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0 with: auto-update-conda: true python-version: "3.12" diff --git a/.github/workflows/debug.yml b/.github/workflows/debug.yml index 45401c704..3fa77be10 100644 --- a/.github/workflows/debug.yml +++ b/.github/workflows/debug.yml @@ -26,7 +26,7 @@ jobs: # - "conda-forge" steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false @@ -71,5 +71,5 @@ jobs: ( for attr in Matrix Scalar Vector Recorder _agg agg base binary descriptor dtypes exceptions expr ffi formatting infix init io lib mask matrix monoid op operator scalar semiring tests unary vector recorder _ss ss ; do echo python -c \"from graphblas import $attr\" ; if ! python -c "from graphblas import $attr" ; then exit 1 ; fi ; done ) - name: Setup tmate session - uses: mxschmitt/action-tmate@v3 + uses: mxschmitt/action-tmate@35b54afac29c97fb54faba5b513f8fbd1882f113 # v3.24 if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }} diff --git a/.github/workflows/imports.yml b/.github/workflows/imports.yml index 7f1596c0a..7b9c4654d 100644 --- a/.github/workflows/imports.yml +++ b/.github/workflows/imports.yml @@ -38,11 +38,11 @@ jobs: # python-version: ["3.11", "3.12", "3.13", "3.14"] # os: ["ubuntu-latest", "macos-latest", "windows-latest"] steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: ${{ needs.rngs.outputs.pyver }} # python-version: ${{ matrix.python-version }} diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 8bceeeadd..567dceb83 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -16,11 +16,11 @@ jobs: name: pre-commit-hooks runs-on: ubuntu-latest steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: "3.11" - - uses: pre-commit/action@v3.0.1 + - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1 diff --git a/.github/workflows/publish_pypi.yml b/.github/workflows/publish_pypi.yml index ad9284f66..5fd126f3e 100644 --- a/.github/workflows/publish_pypi.yml +++ b/.github/workflows/publish_pypi.yml @@ -13,12 +13,12 @@ jobs: shell: bash -l {0} steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false - name: Set up Python - uses: actions/setup-python@v6 + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: "3.11" - name: Install build dependencies @@ -27,7 +27,7 @@ jobs: python -m pip install build twine - name: Build wheel and sdist run: python -m build --sdist --wheel - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: releases path: dist @@ -35,7 +35,7 @@ jobs: - name: Check with twine run: python -m twine check --strict dist/* - name: Publish to PyPI - uses: pypa/gh-action-pypi-publish@v1.13.0 + uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 if: startsWith(github.ref, 'refs/tags/') && github.repository == 'python-graphblas/python-graphblas' with: user: __token__ diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 99fbb580f..309d7a01a 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -105,7 +105,7 @@ jobs: KMP_DUPLICATE_LIB_OK: ${{ contains(matrix.os, 'macos') && 'TRUE' || 'FALSE' }} steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false @@ -130,7 +130,7 @@ jobs: echo "selected: $selected (from: ${choices[*]})" echo "selected=$selected" >> "$GITHUB_OUTPUT" - name: Setup conda - uses: conda-incubator/setup-miniconda@v3 + uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0 id: setup_conda with: auto-update-conda: true @@ -139,7 +139,9 @@ jobs: conda-remove-defaults: ${{ contains(steps.pyver.outputs.selected, 'pypy') && 'false' || 'true' }} channel-priority: ${{ contains(steps.pyver.outputs.selected, 'pypy') && 'flexible' || 'strict' }} activate-environment: graphblas - auto-activate-base: false + # setup-miniconda deprecated ``auto-activate-base`` for this name, and warns + # on every run that still uses the old one. + auto-activate: false - name: Update env run: | # Pick random compatible dependency versions using the version picker script. @@ -369,7 +371,7 @@ jobs: coverage xml coverage report --show-missing - name: codecov - uses: codecov/codecov-action@v5 + uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 with: # Codecov takes tokenless uploads only from forks. A PR from a branch of this # repository, or a push to main, is refused with "Token required because From 9b965bdea1e8ac104f071657c352883318d910bd Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:12:30 -0500 Subject: [PATCH 03/19] Update the pre-commit hooks pre-commit autoupdate: isort 9.0.2, pyupgrade 3.22.0, black 26.10.1, ruff 0.17.1, flake8 7.4.1, prettier 3.9.10 and check-jsonschema 0.38.2, with flake8's hand-pinned plugins moved to flake8-bugbear 26.9.30 and flake8-simplify 0.31.1 (and scripts/check_versions.sh with them). This supersedes pre-commit.ci's #579, which predates black 26.10.1, ruff 0.17 and prettier 3.9.10. pyroma stays on 5.0.1: autoupdate offers 5.1b1, and 5.1 is still in beta on PyPI (5.1b2), which is what the comment above it waits for. None of the new versions changes or flags any file, here or at the top of the current stack. --- .pre-commit-config.yaml | 22 +++++++++++----------- scripts/check_versions.sh | 4 ++-- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index def06f9d5..35a85f72d 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -54,12 +54,12 @@ repos: # We can probably remove `isort` if we come to trust `ruff --fix`, # but we'll need to figure out the configuration to do this in `ruff` - repo: https://github.com/pycqa/isort - rev: 9.0.1 + rev: 9.0.2 hooks: - id: isort # Let's keep `pyupgrade` even though `ruff --fix` probably does most of it - repo: https://github.com/asottile/pyupgrade - rev: v3.21.2 + rev: v3.22.0 hooks: - id: pyupgrade args: [--py311-plus] @@ -69,27 +69,27 @@ repos: - id: auto-walrus args: [--line-length, "100"] - repo: https://github.com/psf/black-pre-commit-mirror - rev: 26.5.1 + rev: 26.10.1 hooks: - id: black - id: black-jupyter - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.16.8 + rev: v0.17.1 hooks: - id: ruff-check args: [--fix-only, --show-fixes] # Let's keep `flake8` even though `ruff` does much of the same. # `flake8-bugbear` and `flake8-simplify` have caught things missed by `ruff`. - repo: https://github.com/PyCQA/flake8 - rev: 7.3.0 + rev: 7.4.1 hooks: - id: flake8 args: ["--config=.flake8"] additional_dependencies: &flake8_dependencies # These versions need updated manually - - flake8==7.3.0 - - flake8-bugbear==26.9.9 - - flake8-simplify==0.30.0 + - flake8==7.4.1 + - flake8-bugbear==26.9.30 + - flake8-simplify==0.31.1 - repo: https://github.com/codespell-project/codespell rev: v2.4.3 hooks: @@ -99,7 +99,7 @@ repos: - tomli; python_version<'3.11' files: ^(graphblas|docs)/ - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.16.8 + rev: v0.17.1 hooks: - id: ruff-check # - id: ruff-format # we prefer black; uncomment to compare with ruff's formatter @@ -122,7 +122,7 @@ repos: hooks: - id: shellcheck - repo: https://github.com/rbubley/mirrors-prettier - rev: v3.9.7 + rev: v3.9.10 hooks: - id: prettier args: [--prose-wrap=preserve] @@ -147,7 +147,7 @@ repos: hooks: - id: actionlint - repo: https://github.com/python-jsonschema/check-jsonschema - rev: 0.38.0 + rev: 0.38.2 hooks: - id: check-dependabot - id: check-github-workflows diff --git a/scripts/check_versions.sh b/scripts/check_versions.sh index f702f2f41..84ea117ad 100755 --- a/scripts/check_versions.sh +++ b/scripts/check_versions.sh @@ -10,8 +10,8 @@ # scripts/audit_pypi_versions.py and scripts/audit_conda_versions.py ask the same # questions automatically, and also check the pools and tables in ci_pick_versions.py # against what is published. The "Audit dependency versions" workflow runs both weekly. -conda search 'flake8-bugbear[channel=conda-forge]>=26.9.9' -conda search 'flake8-simplify[channel=conda-forge]>=0.30.0' +conda search 'flake8-bugbear[channel=conda-forge]>=26.9.30' +conda search 'flake8-simplify[channel=conda-forge]>=0.31.1' conda search 'numpy[channel=conda-forge]>=2.5' # ci_pick_versions.py caps an unpinned pandas below 3.1 for now (see its MAINT note). conda search 'pandas[channel=conda-forge]>=3.0' From 9af0a5edc5ecf556537a67fef93908a8d71069da Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:19:48 -0500 Subject: [PATCH 04/19] Add numba 0.68, awkward 2.14, networkx 3.7 and psg 10.5.1.1 to the CI pools Each is the newest release on conda-forge, and the audits named all four as newer than any pin. * numba 0.68 caps numpy at <2.6, the same as 0.67 (conda-forge's ``depends``). * awkward 2.14 needs awkward-cpp 57 and Python >=3.11, which every Python in the pool has. * networkx 3.7 requires Python >=3.12, so py3.11's pool stops at 3.6. It constrains numpy, scipy and pandas exactly as 3.5 and 3.6 do, so the existing ``>= (3, 5)`` rule covers it; audit_conda_versions.py now checks 3.7's constraints too. * psg 10.5.1.1 is the newest build of SuiteSparse:GraphBLAS 10.5.1, with numpy 2 builds for every Python we test. numpy (2.5), scipy (1.18), pandas (3.0), sparse (0.19) and pyyaml (6.0) have no newer series. scripts/check_versions.sh moves to the new minimums. The rest of the files with dependency versions need no change: pyproject.toml holds floors, environment.yml and dev-requirements.txt are unpinned, and binder/environment.yml and docs/env.yml pin Pythons that are still supported. `ci_pick_versions.py --validate`, audit_pypi_versions.py --samples 200 and audit_conda_versions.py --samples 100 all pass. --- scripts/audit_conda_versions.py | 3 +++ scripts/check_versions.sh | 8 ++++---- scripts/ci_pick_versions.py | 17 ++++++++++------- 3 files changed, 17 insertions(+), 11 deletions(-) diff --git a/scripts/audit_conda_versions.py b/scripts/audit_conda_versions.py index 0d2fe4c16..5b837622d 100755 --- a/scripts/audit_conda_versions.py +++ b/scripts/audit_conda_versions.py @@ -55,6 +55,9 @@ # of these drifts, the matching rule in `apply_constraints` needs another look. # Each entry is (dep, version pin, field, expected constraint). DOCUMENTED_METADATA = [ + ("networkx", "3.7", "constrains", "numpy >=1.25"), + ("networkx", "3.7", "constrains", "scipy >=1.11.2"), + ("networkx", "3.7", "constrains", "pandas >=2.0"), ("networkx", "3.6", "constrains", "numpy >=1.25"), ("networkx", "3.6", "constrains", "scipy >=1.11.2"), ("networkx", "3.6", "constrains", "pandas >=2.0"), diff --git a/scripts/check_versions.sh b/scripts/check_versions.sh index 84ea117ad..05a17b12b 100755 --- a/scripts/check_versions.sh +++ b/scripts/check_versions.sh @@ -16,14 +16,14 @@ conda search 'numpy[channel=conda-forge]>=2.5' # ci_pick_versions.py caps an unpinned pandas below 3.1 for now (see its MAINT note). conda search 'pandas[channel=conda-forge]>=3.0' conda search 'scipy[channel=conda-forge]>=1.18' -conda search 'networkx[channel=conda-forge]>=3.6' -conda search 'awkward[channel=conda-forge]>=2.13' +conda search 'networkx[channel=conda-forge]>=3.7' +conda search 'awkward[channel=conda-forge]>=2.14' conda search 'sparse[channel=conda-forge]>=0.19' # fast_matrix_market is deprecated (no longer maintained; last supported Python is 3.12) -conda search 'numba[channel=conda-forge]>=0.67' +conda search 'numba[channel=conda-forge]>=0.68' conda search 'pyyaml[channel=conda-forge]>=6.0' # matplotlib is installed unpinned for the notebooks job; ci_pick_versions.py caps it # below 3.11 when the numpy pick predates matplotlib's numpy >=1.25 runtime floor. conda search 'matplotlib-base[channel=conda-forge]>=3.11' -conda search 'python-suitesparse-graphblas[channel=conda-forge]>=10.5.0' +conda search 'python-suitesparse-graphblas[channel=conda-forge]>=10.5.1' # conda search 'python[channel=conda-forge]>=3.11 *pypy*' diff --git a/scripts/ci_pick_versions.py b/scripts/ci_pick_versions.py index 8c7d4786c..164ac23ab 100755 --- a/scripts/ci_pick_versions.py +++ b/scripts/ci_pick_versions.py @@ -99,12 +99,12 @@ "3.11": ["2.0", "2.1", "2.2", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", ""], "3.12": ["2.4", "2.5", "2.6", "2.7", "2.8", "2.9", ""], }, - "2.x": ["2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "2.12", "2.13", ""], + "2.x": ["2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "2.12", "2.13", "2.14", ""], } NUMBA_VERSIONS = { "1.x": ["0.57", "0.58", "0.59", "0.60", "0.61", ""], - "2.x": ["0.62", "0.63", "0.64", "0.65", "0.66", "0.67", ""], + "2.x": ["0.62", "0.63", "0.64", "0.65", "0.66", "0.67", "0.68", ""], } # Oldest numba that supports each Python. @@ -130,14 +130,16 @@ "0.65": (2, 5), "0.66": (2, 5), "0.67": (2, 6), + "0.68": (2, 6), } # Deps that only depend on Python version (not numpy) +# networkx 3.7 requires Python >=3.12, so py3.11 stops at 3.6. NETWORKX_VERSIONS = { "3.11": ["2.8", "3.0", "3.1", "3.2", "3.3", "3.4", "3.5", "3.6", ""], - "3.12": ["3.2", "3.3", "3.4", "3.5", "3.6", ""], - "3.13": ["3.4", "3.5", "3.6", ""], - "3.14": ["3.6", ""], + "3.12": ["3.2", "3.3", "3.4", "3.5", "3.6", "3.7", ""], + "3.13": ["3.4", "3.5", "3.6", "3.7", ""], + "3.14": ["3.6", "3.7", ""], } PYYAML_VERSIONS = { @@ -200,6 +202,7 @@ "10.4.0.0", "10.4.1.0", "10.5.0.0", + "10.5.1.1", ] # Oldest psg release that installs on a given Python, by where psg comes from. @@ -388,14 +391,14 @@ def apply_constraints(v, pyver, scipy_pool, numba_pool): # --- networkx / numpy, scipy, pandas floors --- - # conda-forge networkx 3.5 and 3.6 declare `constrains: numpy >=1.25, scipy >=1.11.2, + # conda-forge networkx 3.5 through 3.7 declare `constrains: numpy >=1.25, scipy >=1.11.2, # pandas >=2.0`. conda enforces those against our pins, and since networkx is pinned too # the solve fails outright instead of backing off. Older pins are safe because a pin like # "3.4" can resolve down to 3.4.0, whose floors (numpy >=1.22, scipy >=1.9, pandas >=1.4) # sit at or below everything in our pools. Unpinned networkx is latest, so it gets the # 3.5+ floors. Compare scipy at the minor level: the "1.11" pin resolves to 1.11.4, which # clears the 1.11.2 floor. - # MAINT: 2026-08-04 conda-forge networkx 3.6.1 constrains numpy >=1.25, scipy >=1.11.2 + # MAINT: 2026-10-18 conda-forge networkx 3.6.1 and 3.7 constrain numpy >=1.25, scipy >=1.11.2 if _ver(v["networkx"]) >= (3, 5) and ( _ver(v["numpy"]) < (1, 25) or _ver(v["scipy"]) < (1, 11) or _ver(v["pandas"]) < (2, 0) ): From 4aef6a8e7cdf58417d15b8b325ed941764c38653 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:28:28 -0500 Subject: [PATCH 05/19] Support Python 3.15 conda-forge has Python 3.15.0 final (cp315 and cp315t), and every dependency CI installs has py315 builds: numba 0.68, numpy 2.5.2+, scipy 1.18.1, pandas 3.0.6, awkward 2.13+, psg 10.5.1.1, matplotlib 3.11.2 and coverage 7.16 (compiled, so the tracer works). PyPI has cp315 wheels of all of them except pyyaml, which pip builds from its sdist. Numba supports 3.15 from the start, so this is a single step rather than the usual two: the numba guards in pyproject.toml move to `python_version<'3.16'`. * pyproject.toml: the 3.15 classifier, the numba guards (all five, sparse's included), and py315 in black's target-version (black 26.10 knows it). * test_and_build.yml and imports.yml draw 3.15. The interpreter pin from the update-env step makes it the GIL build (cp315). * ci_pick_versions.py gets a 3.15 row in every per-Python table, and three rules for what has no py315 build: scipy <1.18, pandas <3.0, and awkward <2.13 (awkward is noarch, but each release pins one awkward-cpp, and 56 is the first with py315 builds). numba starts at 0.68 and psg at 10.5.1.1 for every source: its sdist is the first from after psg added 3.15 support. The validator learned the same three rules and now names the floor each failure needs; disabling any one of the new rules makes `--validate` fail. Checked: `--validate` passes (200000 combos), as do audit_pypi_versions.py and audit_conda_versions.py. A dry-run solve of the update-env install, built the way the workflow builds it, succeeds for 60 draws: 3.15 on all five CI platforms with every source and slowtask (numba skipped too), and 3.11-3.14 on four platforms. Every one lands on a GIL interpreter. The full test suite with --runslow passes on 3.15 with the newest pins (numpy 2.5.4, numba 0.68.0, psg 10.5.1.1, scipy 1.18.1, pandas 3.0.6, networkx 3.7, awkward 2.14.0, sparse 0.19.2). --- .github/workflows/imports.yml | 4 +- .github/workflows/test_and_build.yml | 2 +- pyproject.toml | 13 +++--- scripts/ci_pick_versions.py | 60 +++++++++++++++++++++------- 4 files changed, 55 insertions(+), 24 deletions(-) diff --git a/.github/workflows/imports.yml b/.github/workflows/imports.yml index 7b9c4654d..9ceb85ba7 100644 --- a/.github/workflows/imports.yml +++ b/.github/workflows/imports.yml @@ -25,7 +25,7 @@ jobs: - name: RNG for Python version id: pyver run: | - choices=(3.11 3.12 3.13 3.14) + choices=(3.11 3.12 3.13 3.14 3.15) selected=${choices[$((RANDOM % ${#choices[@]}))]} echo "selected: $selected (from: ${choices[*]})" echo "selected=$selected" >> "$GITHUB_OUTPUT" @@ -35,7 +35,7 @@ jobs: # runs-on: ${{ matrix.os }} # strategy: # matrix: - # python-version: ["3.11", "3.12", "3.13", "3.14"] + # python-version: ["3.11", "3.12", "3.13", "3.14", "3.15"] # os: ["ubuntu-latest", "macos-latest", "windows-latest"] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 309d7a01a..1c77e2e63 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -117,7 +117,7 @@ jobs: # We may be able to support pypy if anybody asks for it # 3.9.16 0_73_pypy run: | - choices=(3.11 3.12 3.13 3.14) + choices=(3.11 3.12 3.13 3.14 3.15) selected=${choices[$((RANDOM % ${#choices[@]}))]} echo "selected: $selected (from: ${choices[*]})" echo "selected=$selected" >> "$GITHUB_OUTPUT" diff --git a/pyproject.toml b/pyproject.toml index 9c9701dde..eccf7a417 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -45,6 +45,7 @@ classifiers = [ "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", + "Programming Language :: Python :: 3.15", "Programming Language :: Python :: 3 :: Only", "Intended Audience :: Developers", "Intended Audience :: Other Audience", @@ -62,7 +63,7 @@ dependencies = [ # https://peps.python.org/pep-0771/ (still draft as of 2026-03) # Use e.g. "python-graphblas[suitesparse]" or "python-graphblas[default]" instead "suitesparse-graphblas >=7.4.0.0", - "numba >=0.57; python_version<'3.15'", # make optional where numba is not supported + "numba >=0.57; python_version<'3.16'", # make optional where numba is not supported ] [project.urls] @@ -83,11 +84,11 @@ suitesparse-udf = [ # udf requires numba repr = ["python-graphblas[pandas]"] io = [ "python-graphblas[networkx,scipy]", - "python-graphblas[numba]; python_version<'3.15'", # make optional where numba is not supported + "python-graphblas[numba]; python_version<'3.16'", # make optional where numba is not supported "awkward >=2.0", # sparse needs numba, so it stays optional and takes numba's upper guard from above: # it has to drop out wherever numba does. - "sparse >=0.14; python_version<'3.15'", + "sparse >=0.14; python_version<'3.16'", # fast-matrix-market is no longer maintained; last supported Python is 3.12 ] viz = ["python-graphblas[networkx,scipy]", "matplotlib >=3.6"] @@ -99,11 +100,11 @@ datashade = [ # datashade requires numba test = ["python-graphblas[suitesparse,pandas,scipy]", "packaging >=21", "pytest >=6.2", "tomli >=1"] default = [ "python-graphblas[suitesparse,pandas,scipy]", - "python-graphblas[numba]; python_version<'3.15'", # make optional where numba is not supported + "python-graphblas[numba]; python_version<'3.16'", # make optional where numba is not supported ] all = [ "python-graphblas[default,io,viz,test]", - "python-graphblas[datashade]; python_version<'3.15'", # make optional, b/c datashade needs numba + "python-graphblas[datashade]; python_version<'3.16'", # make optional, b/c datashade needs numba ] [tool.setuptools] @@ -138,7 +139,7 @@ dirty_template = "{tag}+{ccount}.g{sha}.dirty" [tool.black] line-length = 100 -target-version = ["py311", "py312", "py313", "py314"] +target-version = ["py311", "py312", "py313", "py314", "py315"] [tool.isort] sections = ["FUTURE", "STDLIB", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER"] diff --git a/scripts/ci_pick_versions.py b/scripts/ci_pick_versions.py index 164ac23ab..1b271cdde 100755 --- a/scripts/ci_pick_versions.py +++ b/scripts/ci_pick_versions.py @@ -24,7 +24,7 @@ # # When updating versions here, also update scripts/check_versions.sh # -# To add a Python version (3.15, say): add it to `choices` in the "RNG for Python +# To add a Python version (3.16, say): add it to `choices` in the "RNG for Python # version" step of test_and_build.yml, give it a row in every table keyed by Python # below (`check_python_pools` fails the run if you miss one), and revisit the floors # in `apply_constraints`. Those floors are written as `_ver(pyver) >= (3, N)`, so a @@ -32,12 +32,14 @@ # enough to keep CI honest, but usually still too loose to install, so check each dep. # --------------------------------------------------------------------------- -# numpy 2.5 requires Python >=3.12, so py3.11 stops at 2.4. +# numpy 2.5 requires Python >=3.12, so py3.11 stops at 2.4. conda-forge's first py3.15 +# builds are numpy 2.5.2, which the "2.5" pin resolves to. NUMPY_VERSIONS = { "3.11": ["1.24", "1.25", "1.26", "2.0", "2.1", "2.2", "2.3", "2.4", ""], "3.12": ["1.26", "2.0", "2.1", "2.2", "2.3", "2.4", "2.5", ""], "3.13": ["2.1", "2.2", "2.3", "2.4", "2.5", ""], "3.14": ["2.3", "2.4", "2.5", ""], + "3.15": ["2.5", ""], } # Deps that depend on numpy version (1.x vs 2.x path). @@ -108,7 +110,13 @@ } # Oldest numba that supports each Python. -NUMBA_MIN_PYTHON = {"3.11": (0, 57), "3.12": (0, 59), "3.13": (0, 61), "3.14": (0, 63)} +NUMBA_MIN_PYTHON = { + "3.11": (0, 57), + "3.12": (0, 59), + "3.13": (0, 61), + "3.14": (0, 63), + "3.15": (0, 68), +} # The numpy ceiling each numba pin declares. numba caps `numpy =3.9), so hold # them to the wheel floor rather than test a build psg never claimed works. -# MAINT: 2026-09-11 checked conda-forge build strings and PyPI wheel tags for psg 9-10 +# +# py3.15 starts at 10.5.1.1 everywhere: it is psg's first release with py315 builds on +# conda-forge and cp315 wheels on PyPI, and its sdist is the first from after psg added +# Python 3.15 support. +# MAINT: 2026-10-18 checked conda-forge build strings and PyPI wheel tags for psg 9-10 PSG_MIN_PYTHON = { "3.14": {"conda-forge": "10.0.1.1", "wheel": "10.3.1.0", "source": "10.3.1.0"}, + "3.15": {"conda-forge": "10.5.1.1", "wheel": "10.5.1.1", "source": "10.5.1.1"}, } @@ -323,8 +339,10 @@ def apply_constraints(v, pyver, scipy_pool, numba_pool): # --- scipy / Python version availability --- - # scipy <1.14 has no py3.13 builds; scipy <1.16 has no py3.14 builds - if _ver(pyver) >= (3, 14) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 16): + # scipy <1.14 has no py3.13 builds, <1.16 none for py3.14, and <1.18 none for py3.15 + if _ver(pyver) >= (3, 15) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 18): + v["scipy"] = _pick_scipy(pyver, npver, scipy_pool, floor=(1, 18)) + elif _ver(pyver) >= (3, 14) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 16): v["scipy"] = _pick_scipy(pyver, npver, scipy_pool, floor=(1, 16)) elif _ver(pyver) >= (3, 13) and v["scipy"] == "1.13": v["scipy"] = _pick_scipy(pyver, npver, scipy_pool, floor=(1, 14)) @@ -335,8 +353,10 @@ def apply_constraints(v, pyver, scipy_pool, numba_pool): # --- pandas constraints --- - # pandas <2.3 has no py3.14 builds - if _ver(pyver) >= (3, 14) and v["pandas"] == "2.2": + # pandas <2.3 has no py3.14 builds, and <3.0 none for py3.15 + if _ver(pyver) >= (3, 15) and v["pandas"] not in ("", "NA") and _ver(v["pandas"]) < (3, 0): + v["pandas"] = random.choice(["3.0", ""]) + elif _ver(pyver) >= (3, 14) and v["pandas"] == "2.2": v["pandas"] = random.choice(["2.3", "3.0", ""]) # pandas 3.0 requires scipy >=1.14.1. It also requires numba >=0.60, which needs no @@ -370,8 +390,12 @@ def apply_constraints(v, pyver, scipy_pool, numba_pool): # --- awkward / Python version availability --- - # awkward <2.7 has no py3.13 builds; awkward <2.8 has no py3.14 builds - if _ver(pyver) >= (3, 14) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 8): + # awkward <2.7 has no py3.13 builds, <2.8 none for py3.14, and <2.13 none for py3.15 + # (awkward is noarch, but each release pins one awkward-cpp, and awkward-cpp 56 is the + # first with py315 builds) + if _ver(pyver) >= (3, 15) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 13): + v["awkward"] = random.choice([a for a in AWKWARD_VERSIONS["2.x"] if _ver(a) >= (2, 13)]) + elif _ver(pyver) >= (3, 14) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 8): v["awkward"] = random.choice(["2.8", "2.9", ""]) elif _ver(pyver) >= (3, 13) and v["awkward"] == "2.6": v["awkward"] = random.choice(["2.7", "2.8", "2.9", ""]) @@ -688,8 +712,10 @@ def validate(v, pyver, source_type=None): errors.append(f"scipy {v['scipy']} requires numpy <{cap}, got {v['numpy'] or 'latest'}") # scipy Python availability - if _ver(pyver) >= (3, 14) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 16): - errors.append(f"scipy {v['scipy']} has no py{pyver} build") + if _ver(pyver) >= (3, 15) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 18): + errors.append(f"scipy {v['scipy']} has no py{pyver} build (needs >=1.18)") + elif _ver(pyver) >= (3, 14) and v["scipy"] not in ("", "NA") and _ver(v["scipy"]) < (1, 16): + errors.append(f"scipy {v['scipy']} has no py{pyver} build (needs >=1.16)") elif _ver(pyver) >= (3, 13) and v["scipy"] == "1.13": errors.append(f"scipy 1.13 has no py{pyver} build") if v["scipy"] and _ver(pyver) < (floor := SCIPY_MIN_PYTHON.get(v["scipy"], (0,))): @@ -697,7 +723,9 @@ def validate(v, pyver, source_type=None): errors.append(f"scipy {v['scipy']} requires Python >={need}, has no py{pyver} build") # pandas Python availability - if _ver(pyver) >= (3, 14) and v["pandas"] == "2.2": + if _ver(pyver) >= (3, 15) and v["pandas"] not in ("", "NA") and _ver(v["pandas"]) < (3, 0): + errors.append(f"pandas {v['pandas']} has no py{pyver} build") + elif _ver(pyver) >= (3, 14) and v["pandas"] == "2.2": errors.append(f"pandas 2.2 has no py{pyver} build") # pandas 3.0 requirements (numba >=0.60 is unreachable; see `apply_constraints`) @@ -714,8 +742,10 @@ def validate(v, pyver, source_type=None): errors.append(f"pandas {v['pandas']} constrains scipy >=1.10.0, got {v['scipy']}") # awkward Python availability - if _ver(pyver) >= (3, 14) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 8): - errors.append(f"awkward {v['awkward']} has no py{pyver} build") + if _ver(pyver) >= (3, 15) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 13): + errors.append(f"awkward {v['awkward']} has no py{pyver} build (needs >=2.13)") + elif _ver(pyver) >= (3, 14) and v["awkward"] not in ("", "NA") and _ver(v["awkward"]) < (2, 8): + errors.append(f"awkward {v['awkward']} has no py{pyver} build (needs >=2.8)") elif _ver(pyver) >= (3, 13) and v["awkward"] == "2.6": errors.append(f"awkward 2.6 has no py{pyver} build") From 8d710530e8f8be802159461ae91aaddd4b1bc4c7 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:32:05 -0500 Subject: [PATCH 06/19] Run zizmor in pre-commit and CI, and fix what it reports zizmor audits GitHub Actions workflows for security problems. It runs as a pre-commit hook (and so on pre-commit.ci, offline), and security.yml runs it again with a token for its online audits: advisories against the actions we use, pins to commits outside an action's repository, and version comments that no longer match their pin. security.yml also runs pysentry, which pre-commit.ci skips for lack of network. Both run on PRs that touch pyproject.toml, .pre-commit-config.yaml or .github/, and weekly for advisories published since. The setup follows afar's. pysentry now runs when pyproject.toml or .pre-commit-config.yaml changes rather than at every commit, where it fails offline, and with ``--direct-only`` it audits the floors in pyproject.toml instead of the newest versions uv resolves, which would hide a floor that allows a vulnerable release. It finds nothing either way today. What zizmor reported, and what changed: * excessive-permissions: test_and_build.yml, imports.yml, debug.yml and publish_pypi.yml had no ``permissions:``, so their jobs got this repository's default token, which is read-write on everything. They now ask for ``contents: read``, which is all checkout needs; uploads to Codecov and of artifacts do not use the token. * template-injection: the "Update env" and "Build extension module" steps pasted step outputs into their scripts with ``${{ }}``. The values come from fixed lists, so nothing could be injected, but they now reach the scripts as environment variables. That includes the ``format('python={0} python-gil', ...)`` spec, which zizmor does not flag but which is the same pattern. * use-trusted-publishing: suppressed inline on the token, with a TODO. A ``pypi`` environment with required reviewers already exists, but moving to trusted publishing needs a publisher registered on PyPI, which cannot be checked from here. * unpinned-uses and dependabot-cooldown were fixed by the previous commits. .github/zizmor.yml ignored all three of the first findings for whole files; nothing is left for it to ignore, so it is gone. Offline and online, zizmor now reports no findings (one ignored, the inline suppression). The default persona is the one afar uses; the pedantic one would also flag the matrix values in run steps, which are not attacker-controlled. --- .github/workflows/debug.yml | 3 ++ .github/workflows/imports.yml | 3 ++ .github/workflows/publish_pypi.yml | 8 +++- .github/workflows/security.yml | 58 ++++++++++++++++++++++++++++ .github/workflows/test_and_build.yml | 30 ++++++++++---- .github/zizmor.yml | 16 -------- .pre-commit-config.yaml | 25 +++++++++--- 7 files changed, 112 insertions(+), 31 deletions(-) create mode 100644 .github/workflows/security.yml delete mode 100644 .github/zizmor.yml diff --git a/.github/workflows/debug.yml b/.github/workflows/debug.yml index 3fa77be10..bb8503570 100644 --- a/.github/workflows/debug.yml +++ b/.github/workflows/debug.yml @@ -9,6 +9,9 @@ on: required: false default: false +permissions: + contents: read + jobs: build: runs-on: ubuntu-latest diff --git a/.github/workflows/imports.yml b/.github/workflows/imports.yml index 9ceb85ba7..9b719ccb6 100644 --- a/.github/workflows/imports.yml +++ b/.github/workflows/imports.yml @@ -6,6 +6,9 @@ on: branches: - main +permissions: + contents: read + jobs: rngs: runs-on: ubuntu-latest diff --git a/.github/workflows/publish_pypi.yml b/.github/workflows/publish_pypi.yml index 5fd126f3e..63e4d9427 100644 --- a/.github/workflows/publish_pypi.yml +++ b/.github/workflows/publish_pypi.yml @@ -5,6 +5,9 @@ on: tags: - "20*" +permissions: + contents: read + jobs: build_and_deploy: runs-on: ubuntu-latest @@ -34,9 +37,12 @@ jobs: if-no-files-found: error - name: Check with twine run: python -m twine check --strict dist/* + # TODO: publish with Trusted Publishing instead of a token. That needs a trusted + # publisher registered for this repository on PyPI, which this file cannot check, + # and a release is the wrong time to find out it is missing. - name: Publish to PyPI uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 if: startsWith(github.ref, 'refs/tags/') && github.repository == 'python-graphblas/python-graphblas' with: user: __token__ - password: ${{ secrets.PYPI_TOKEN }} + password: ${{ secrets.PYPI_TOKEN }} # zizmor: ignore[use-trusted-publishing] diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml new file mode 100644 index 000000000..e7840ac2c --- /dev/null +++ b/.github/workflows/security.yml @@ -0,0 +1,58 @@ +name: Security + +# Run the security hooks of .pre-commit-config.yaml, with its versions and arguments, where +# pre-commit.ci cannot: with network access, and weekly, for advisories published since. +on: + pull_request: + paths: + - pyproject.toml + - .pre-commit-config.yaml + - .github/** + schedule: + # Mondays at 06:41 UTC + - cron: "41 6 * * 1" + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + pysentry: + name: pysentry + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.14" + # Advisories against the floors in pyproject.toml + - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1 + with: + extra_args: pysentry --all-files + + zizmor: + name: zizmor + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.14" + # With a token, zizmor also runs its online audits, such as advisories against the actions + # these workflows use (known-vulnerable-actions) and pins to commits outside an action's + # repository (impostor-commit). + - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1 + env: + GH_TOKEN: ${{ github.token }} + with: + extra_args: zizmor --all-files diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 1c77e2e63..402a8cd15 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -17,6 +17,9 @@ on: branches: - main +permissions: + contents: read + # concurrency: # group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} # cancel-in-progress: true @@ -143,15 +146,20 @@ jobs: # on every run that still uses the old one. auto-activate: false - name: Update env + env: + # Step outputs reach the script through the environment, not ``${{ }}``, so + # bash never parses them as code (zizmor's template-injection audit). + PYVER: ${{ steps.pyver.outputs.selected }} + SOURCETYPE: ${{ steps.sourcetype.outputs.selected }} run: | # Pick random compatible dependency versions using the version picker script. # See scripts/ci_pick_versions.py for version pools and constraint logic. - pyver=$(python -c "v='${{ steps.pyver.outputs.selected }}'.split('.');print(f'{v[0]}.{v[1]}')") + pyver=$(python -c "import os; v=os.environ['PYVER'].split('.');print(f'{v[0]}.{v[1]}')") # Assign first, then eval. `eval "$(cmd)"` reports eval's status, not cmd's, and # this step's shell has no `-e`, so a picker that exits non-zero (a pool table it # rejects, say) would leave every *ver variable unset and quietly install the # latest of everything: a green run that tested nothing it claims to have tested. - if ! picks=$(python scripts/ci_pick_versions.py --python "$pyver" --source ${{ steps.sourcetype.outputs.selected }}) ; then + if ! picks=$(python scripts/ci_pick_versions.py --python "$pyver" --source "$SOURCETYPE") ; then echo "::error::ci_pick_versions.py failed; refusing to install unpinned dependencies" exit 1 fi @@ -160,7 +168,7 @@ jobs: # Set up psg conda package variable (conda-forge installs via conda, others via pip). # For conda-forge, psg must always be installed; an empty psgver just means "latest". psg="" - if [[ ${{ steps.sourcetype.outputs.selected}} == "conda-forge" ]] ; then + if [[ $SOURCETYPE == "conda-forge" ]] ; then psg=python-suitesparse-graphblas${psgver} fi @@ -216,6 +224,11 @@ jobs: # python-suitesparse-graphblas pins the same pair in # .github/workflows/test.yml, and asks for python-freethreading on the # matrix entries that want the other ABI. + if [[ $PYVER == *pypy* ]] ; then + interpreter=pypy + else + interpreter="python=$PYVER python-gil" + fi $(command -v mamba || command -v conda) install -c nodefaults \ packaging pip pytest coverage pytest-randomly cffi donfig tomli c-compiler make \ pyyaml${yamlver} ${sparse} pandas${pdver} scipy${spver} numpy${npver} ${awkward} \ @@ -224,29 +237,30 @@ jobs: ${{ matrix.slowtask == 'notebooks' && '"matplotlib${mplver}" nbconvert jupyter "ipython>=7" drawsvg' || '' }} \ ${{ steps.sourcetype.outputs.selected == 'upstream' && 'cython' || '' }} \ ${{ steps.sourcetype.outputs.selected != 'wheel' && '"graphblas>=7.4"' || '' }} \ - ${{ contains(steps.pyver.outputs.selected, 'pypy') && 'pypy' || '' }} \ - ${{ !contains(steps.pyver.outputs.selected, 'pypy') && format('python={0} python-gil', steps.pyver.outputs.selected) || '' }} \ + ${interpreter} \ ${{ matrix.os == 'windows-latest' && 'cmake' || 'm4' }} \ # ${{ matrix.os != 'windows-latest' && 'pytest-forked' || '' }} # to investigate crashes - name: Build extension module + env: + SOURCETYPE: ${{ steps.sourcetype.outputs.selected }} run: | # ``python -m pip`` ensures the conda env's interpreter drives the # install, not a system ``pip`` on PATH. - if [[ ${{ steps.sourcetype.outputs.selected }} == "wheel" ]]; then + if [[ $SOURCETYPE == "wheel" ]]; then # Add --pre if installing a pre-release python -m pip install --no-deps --only-binary ":all:" suitesparse-graphblas${psgver} # Add the below line to the conda install command above if installing from test.pypi.org # ${{ steps.sourcetype.outputs.selected == 'wheel' && 'setuptools setuptools-git-versioning wheel cython' || '' }} \ # python -m pip install --no-deps --only-binary ":all:" --index-url https://test.pypi.org/simple/ "suitesparse-graphblas>=7.4.3" - elif [[ ${{ steps.sourcetype.outputs.selected }} == "source" ]]; then + elif [[ $SOURCETYPE == "source" ]]; then # Add --pre if installing a pre-release python -m pip install --no-deps --no-binary suitesparse-graphblas suitesparse-graphblas${psgver} # Add the below line to the conda install command above if installing from test.pypi.org # ${{ steps.sourcetype.outputs.selected == 'source' && 'setuptools setuptools-git-versioning wheel cython' || '' }} \ # python -m pip install --no-deps --no-build-isolation --no-binary suitesparse-graphblas --index-url https://test.pypi.org/simple/ suitesparse-graphblas==7.4.3.3 - elif [[ ${{ steps.sourcetype.outputs.selected }} == "upstream" ]]; then + elif [[ $SOURCETYPE == "upstream" ]]; then python -m pip install --no-deps git+https://github.com/GraphBLAS/python-suitesparse-graphblas.git@main#egg=suitesparse-graphblas fi python -m pip install --no-deps -e . diff --git a/.github/zizmor.yml b/.github/zizmor.yml deleted file mode 100644 index 61f32c2e0..000000000 --- a/.github/zizmor.yml +++ /dev/null @@ -1,16 +0,0 @@ -rules: - use-trusted-publishing: - # TODO: we should update to use trusted publishing - ignore: - - publish_pypi.yml - excessive-permissions: - # It is probably good practice to use narrow permissions - ignore: - - debug.yml - - imports.yml - - publish_pypi.yml - - test_and_build.yml - template-injection: - # We use templates pretty heavily - ignore: - - test_and_build.yml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 35a85f72d..dc1695667 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -11,7 +11,9 @@ ci: autoupdate_commit_msg: "chore: update pre-commit hooks" autofix_commit_msg: "style: pre-commit fixes" # pysentry downloads advisory databases at run time, and pre-commit.ci - # runs hooks without network access, so it can only run locally. + # runs hooks without network access, so it runs locally and in security.yml. + # zizmor does run on pre-commit.ci, offline, which skips its online audits; + # security.yml runs those. skip: [pylint, no-commit-to-branch, pysentry] fail_fast: false default_language_version: @@ -146,6 +148,10 @@ repos: rev: v1.7.12 hooks: - id: actionlint + - repo: https://github.com/zizmorcore/zizmor-pre-commit + rev: v1.31.1 + hooks: + - id: zizmor - repo: https://github.com/python-jsonschema/check-jsonschema rev: 0.38.2 hooks: @@ -156,15 +162,22 @@ repos: rev: v1.38.0 hooks: - id: yamllint - # - repo: https://github.com/woodruffw/zizmor-pre-commit - # rev: v1.14.2 - # hooks: - # - id: zizmor - repo: https://github.com/pysentry/pysentry-pre-commit rev: v0.5.0 hooks: - id: pysentry - args: ["--format=human", "--fail-on=low", "--sources=pypa,osv,pypi"] + # Run when pyproject.toml or this file changes rather than at every commit, where + # it fails offline. security.yml also runs it weekly. + always_run: false + files: ^(pyproject\.toml|\.pre-commit-config\.yaml)$ + # Audit the floors in pyproject.toml. Without `--direct-only`, pysentry resolves the + # newest versions with uv (which the hook installs), and no longer sees a floor that + # allows a vulnerable release. + args: + - --format=human + - --fail-on=low + - --sources=pypa,osv,pypi + - --direct-only - repo: meta hooks: - id: check-hooks-apply From bc364c57eae2396524a85603367939fb4c2f861e Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Sun, 18 Oct 2026 00:32:32 -0500 Subject: [PATCH 07/19] Re-date the numba#5395 notes: bool pointers still fail on Numba 0.68 The two MAINT notes on the BOOL-to-INT8 routing asked for a re-test. A cfunc that reads ``CPointer(boolean)`` still fails with "cond is not i1: i8", and one that writes it with "cannot store i1 to i8*", on Numba 0.66 and on 0.68, the release that adds Python 3.15. The routing stays; only the notes' dates and versions change. --- graphblas/core/operator/udf.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/graphblas/core/operator/udf.py b/graphblas/core/operator/udf.py index ab8144fc2..30b3e7ff4 100644 --- a/graphblas/core/operator/udf.py +++ b/graphblas/core/operator/udf.py @@ -39,7 +39,7 @@ def _bool_to_int8(dtype): numba/numba#5395. Routing BOOL through INT8 sidesteps that; GraphBLAS coerces back at the cfunc boundary. - MAINT 2026-05-24: still hits on Numba 0.65. Re-test periodically and + MAINT 2026-10-18: still hits on Numba 0.68. Re-test periodically and drop the INT8 routing when upstream is fixed. """ return INT8 if dtype == BOOL else dtype @@ -306,7 +306,7 @@ def _input_operand(dtype, var): if dtype == BOOL: # Numba can't compile bool ptrs (numba/numba#5395); expose them # as int8 and cast on deref. - # MAINT 2026-05-24: still hits on Numba 0.65; re-test periodically. + # MAINT 2026-10-18: still hits on Numba 0.68; re-test periodically. return "", f"bool({var}_ptr[0])", nt.CPointer(INT8.numba_type) return "", f"{var}_ptr[0]", nt.CPointer(dtype.numba_type) From 0884f02736dff84b7005105dbf629d5cc6fe23e0 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Mon, 19 Oct 2026 02:27:53 -0500 Subject: [PATCH 08/19] Publish to PyPI with Trusted Publishing The publish step used a PyPI API token stored as a secret. A trusted publisher is now registered on PyPI for this repository, so the upload moves to its own job that exchanges a short-lived OIDC token instead: only that job gets `id-token: write`, and it runs in the `pypi` environment, which the PyPI registration names and which asks a maintainer to approve the upload. The build job is unchanged and hands the files over as the `releases` artifact it already uploads. This removes zizmor's last inline suppression. --- .github/workflows/publish_pypi.yml | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/.github/workflows/publish_pypi.yml b/.github/workflows/publish_pypi.yml index 63e4d9427..3370664f9 100644 --- a/.github/workflows/publish_pypi.yml +++ b/.github/workflows/publish_pypi.yml @@ -37,12 +37,21 @@ jobs: if-no-files-found: error - name: Check with twine run: python -m twine check --strict dist/* - # TODO: publish with Trusted Publishing instead of a token. That needs a trusted - # publisher registered for this repository on PyPI, which this file cannot check, - # and a release is the wrong time to find out it is missing. + + publish: + needs: build_and_deploy + if: startsWith(github.ref, 'refs/tags/') && github.repository == 'python-graphblas/python-graphblas' + runs-on: ubuntu-latest + # The trusted publisher registered on PyPI names this environment, which also asks a + # maintainer to approve the upload. + environment: pypi + permissions: + # Only this job can mint the OIDC token that PyPI exchanges for upload rights. + id-token: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: releases + path: dist - name: Publish to PyPI uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 - if: startsWith(github.ref, 'refs/tags/') && github.repository == 'python-graphblas/python-graphblas' - with: - user: __token__ - password: ${{ secrets.PYPI_TOKEN }} # zizmor: ignore[use-trusted-publishing] From a7b44f089fe032f1cf2c2e9422f05638db5eed57 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Mon, 19 Oct 2026 02:54:53 -0500 Subject: [PATCH 09/19] Leave py315 out of black's target versions for now CI's bizarro job installs black from conda-forge, and on Python 3.15 the newest black it can install there is 26.1, because the pytokens >=0.4 that black 26.5 needs has no 3.15 build yet. Black 26.1 does not know "py315" and exits with an error, so the auto-generated code check failed on every 3.15 draw. Any other black older than 26.5 refused to run on this repository for the same reason. Nothing here uses syntax new in 3.15, so the formatting is the same; a MAINT note says when to add it back. --- pyproject.toml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index eccf7a417..cf0251968 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -139,7 +139,11 @@ dirty_template = "{tag}+{ccount}.g{sha}.dirty" [tool.black] line-length = 100 -target-version = ["py311", "py312", "py313", "py314", "py315"] +# MAINT: add "py315" once a black that knows it (26.5 or later) installs on Python 3.15 +# from conda-forge (2026-10-19: its pytokens >=0.4 has no 3.15 build, so 3.15 gets black +# 26.1, which rejects "py315" and stops CI's auto-generated code check). Listing it also +# makes any older black refuse to run on this repository. +target-version = ["py311", "py312", "py313", "py314"] [tool.isort] sections = ["FUTURE", "STDLIB", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER"] From c6a2d47846a9ea71737829941d6fba5e9e74484e Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Thu, 22 Oct 2026 10:56:04 -0500 Subject: [PATCH 10/19] Install the drawn psg version on wheel and source draws The version picker draws a python-suitesparse-graphblas version for every source type, but ``psgver`` was a shell variable of the "Update env" step. "Build extension module", which pip-installs psg for wheel and source draws, runs in a new shell, where ``suitesparse-graphblas${psgver}`` expanded to plain ``suitesparse-graphblas``. So every wheel and source draw installed the newest psg on PyPI, whatever the "versions:" line in its log said; only conda-forge draws, which install psg in "Update env" itself, got the version they drew. It has been this way since wheel and source draws began in #385. It turned up in the CI logs read for why test_ssjit.py's JIT tests skip in CI. In all four wheel and source jobs there that drew a pin (psg 8.0.2.1, 9.4.5.0, 10.1.1.0 and 10.3.1.0), pip installed 10.5.1.1. "Update env" now passes psgver on as a step output, and "Build extension module" reads it through ``env:`` like the other step outputs (zizmor's template-injection audit). An output rather than ``$GITHUB_ENV``, so no other step sees it. Installing the drawn version brings two kinds of draw to life for the first time, and neither could have worked: * source: psg's sdist compiles against conda-forge's GraphBLAS, which was always the newest one (``graphblas>=7.4``). psg 9.4.5.0's sdist does not compile against GraphBLAS 10.5.1 (undeclared GxB_*_GPU_ID constants, changed GxB_Iterator and *_get signatures); against 9.4.5 it builds and imports. A source draw now asks for the GraphBLAS that conda-forge's build of the same psg depends on: python-suitesparse-graphblas X.Y.Z.W requires graphblas >=X.Y.Z,> "$GITHUB_OUTPUT" + + # Wheels bundle their own GraphBLAS. A source draw compiles psg's sdist against + # conda-forge's, so give it the one conda-forge pairs with that psg release: + # python-suitesparse-graphblas X.Y.Z.W depends on graphblas >=X.Y.Z,=7" drawsvg' || '' }} \ ${{ steps.sourcetype.outputs.selected == 'upstream' && 'cython' || '' }} \ - ${{ steps.sourcetype.outputs.selected != 'wheel' && '"graphblas>=7.4"' || '' }} \ + ${graphblas} \ ${interpreter} \ ${{ matrix.os == 'windows-latest' && 'cmake' || 'm4' }} \ # ${{ matrix.os != 'windows-latest' && 'pytest-forked' || '' }} # to investigate crashes - name: Build extension module env: SOURCETYPE: ${{ steps.sourcetype.outputs.selected }} + PSGVER: ${{ steps.update_env.outputs.psgver }} run: | # ``python -m pip`` ensures the conda env's interpreter drives the # install, not a system ``pip`` on PATH. if [[ $SOURCETYPE == "wheel" ]]; then # Add --pre if installing a pre-release - python -m pip install --no-deps --only-binary ":all:" suitesparse-graphblas${psgver} + python -m pip install --no-deps --only-binary ":all:" "suitesparse-graphblas${PSGVER}" # Add the below line to the conda install command above if installing from test.pypi.org # ${{ steps.sourcetype.outputs.selected == 'wheel' && 'setuptools setuptools-git-versioning wheel cython' || '' }} \ # python -m pip install --no-deps --only-binary ":all:" --index-url https://test.pypi.org/simple/ "suitesparse-graphblas>=7.4.3" elif [[ $SOURCETYPE == "source" ]]; then # Add --pre if installing a pre-release - python -m pip install --no-deps --no-binary suitesparse-graphblas suitesparse-graphblas${psgver} + python -m pip install --no-deps --no-binary suitesparse-graphblas "suitesparse-graphblas${PSGVER}" # Add the below line to the conda install command above if installing from test.pypi.org # ${{ steps.sourcetype.outputs.selected == 'source' && 'setuptools setuptools-git-versioning wheel cython' || '' }} \ diff --git a/scripts/ci_pick_versions.py b/scripts/ci_pick_versions.py index 1b271cdde..2d7c296e8 100755 --- a/scripts/ci_pick_versions.py +++ b/scripts/ci_pick_versions.py @@ -195,8 +195,10 @@ ], "3.12": ["8.2.0.1", "8.2.1.0"], }, + # The picker does not know the OS, and 7.4.3.2's wheels are manylinux x86_64 only, + # so a "wheel" draw of it would fail to install on every other runner. "wheel": { - "3.11": ["7.4.3.2", "8.0.2.1", "8.2.0.1", "8.2.1.0"], + "3.11": ["8.0.2.1", "8.2.0.1", "8.2.1.0"], "3.12": ["8.2.0.1", "8.2.1.0"], }, } From 9522cb2abf4e6af4d70e4218d8ee69d0aca116ad Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Thu, 22 Oct 2026 11:03:00 -0500 Subject: [PATCH 11/19] Run the JIT tests with --runslow in every Linux and macOS notebooks job Most of test_ssjit.py needs four things in one pytest run: the suitesparse backend, --runslow, Numba, and a GraphBLAS that can JIT-compile. The backend is drawn per pass, --runslow is on in one pass of each pytest job and in neither pass of a notebooks job, Numba is skipped at random outside the notebooks jobs, and psg's wheels never JIT. In one run in four or five no job had all four, so codecov reported about 210 lines of test_ssjit.py as missed on #591 even though the JIT works in CI. That came from reading CI logs for why those tests skip. The C compiler is fine: every Linux and macOS job on conda-forge's GraphBLAS (conda-forge, source and upstream draws) passes the JIT probe, and scripts/jit_diagnostics.py prints "Result: True". The skips come from psg's wheels, which build GraphBLAS with GRAPHBLAS_USE_JIT=OFF, and from Windows, where conda-forge's GraphBLAS names the build machine's cl.exe. A new step runs test_ssjit.py on its own, under coverage, with --runslow and --backend=suitesparse, in the ubuntu-latest and macos-latest notebooks jobs, before the coverage report and upload. Those jobs install Numba unless the version picker rules it out (some source draws on Python 3.11 and 3.12), and draw psg from all four sources. It mirrors the Windows-only "More tests for coverage" step, and runs the file in order (``-p no:randomly``), as local runs do. No condition on Numba or the JIT: where either is missing, the tests that need it skip themselves and the rest still run and count, so the step passes. Locally (macOS arm64), the step's command: * psg 10.5 with Numba and a working JIT: 48 passed, in about 45 s. * The same with Numba blocked: 19 passed, 29 skipped. * conda-forge's psg 9.3.1.0, 8.2.1.0 and 8.0.2.1, each with Numba and c-compiler as a notebooks job installs them: 47, 32 and 32 passed, the rest skipped for needing a newer SuiteSparse:GraphBLAS. * psg's 8.0.2.1 and 9.3.1.0 wheels, without Numba: all passed or skipped. --- .github/workflows/test_and_build.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index ed3add816..8e0c67486 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -390,6 +390,17 @@ jobs: coverage run -a -m pytest --color=yes --runslow --no-mapnumpy -p no:randomly -v -k 'test_commutes or test_bool_doesnt_get_too_large or test_npbinary or test_npmonoid or test_npsemiring' coverage run -a -m pytest --color=yes --runslow --mapnumpy -p no:randomly -k 'test_bool_doesnt_get_too_large or test_npunary or test_binaryop_monoid_numpy' coverage run -a -m pytest --color=yes -x --no-mapnumpy --runslow -k test_binaryop_attributes_numpy graphblas/tests/test_op.py + - name: JIT tests for coverage + if: matrix.slowtask == 'notebooks' && matrix.os != 'windows-latest' + run: | + # Most of test_ssjit.py needs the suitesparse backend, --runslow, Numba and a + # GraphBLAS that can JIT-compile, all in one pytest run. In one run in four or five, + # no job draws all four, so its coverage came and went. 'notebooks' jobs install + # Numba unless the picker rules it out. Windows is left out: conda-forge's GraphBLAS + # there names the build machine's compiler, so it never compiles. The tests skip + # what a draw cannot do: the JIT tests on psg's wheels (built without the JIT) and + # on GraphBLAS 7, the UDT tests without Numba. + coverage run -a -m pytest --color=yes --runslow --backend=suitesparse -p no:randomly -v graphblas/tests/test_ssjit.py - name: Auto-generated code check if: matrix.slowtask == 'pytest_bizarro' run: | From 791c08a0b4702d6cb10e58e716c1b93fca285864 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Thu, 22 Oct 2026 10:58:17 -0500 Subject: [PATCH 12/19] Cancel a CI run's identical twin, keyed on the PR's head commit When one push moves both a PR's head and its base branch, GitHub often delivers that PR's pull_request event twice, and Tests and Test imports each ran twice on the same merge commit. Restacking this repository's PR stack moves every branch in one push: in the last three restacks, 37 PRs had their base moved each time, and 48 of those 111 PR updates got a duplicate run (15, 13 and 20), the twins arriving up to about 50 s apart. It is GitHub's doing, not the push: gh-stack pushes once, with one ``git push`` and no API call after it, and an ``--atomic`` push got the same duplicates. The PRs that got one show ``base_ref_force_pushed`` beside ``head_ref_force_pushed`` on their timelines. This came out of tracing the 40 queued duplicates left after the last restack. The Security workflow, whose concurrency group cancels the first twin within seconds, never kept one. Both workflows now have a concurrency group keyed on the workflow, the PR number and the PR's head commit, with cancel-in-progress, so the later twin cancels the earlier one, which is nearly always still queued: * A run for an earlier commit of the same PR is in another group and is left to finish, as before. The group test_and_build.yml had commented out since #459, keyed on the branch, would cancel it; this replaces it. * A push to main has no pull_request, so the group falls back to the run id and the pushed commit: each push's run is a group of its own and is never cancelled. Neither workflow has another trigger; workflow_dispatch would key on its run id the same way. * A re-run keeps its run's group, so re-running a cancelled twin while the other is still going cancels that one. --- .github/workflows/imports.yml | 5 +++++ .github/workflows/test_and_build.yml | 11 ++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/.github/workflows/imports.yml b/.github/workflows/imports.yml index 9b719ccb6..9adfb6b0e 100644 --- a/.github/workflows/imports.yml +++ b/.github/workflows/imports.yml @@ -9,6 +9,11 @@ on: permissions: contents: read +# Cancels a run's identical twin only; see the note in test_and_build.yml. +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-${{ github.event.pull_request.head.sha || github.sha }} + cancel-in-progress: true + jobs: rngs: runs-on: ubuntu-latest diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 8e0c67486..0f430629f 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -20,9 +20,14 @@ on: permissions: contents: read -# concurrency: -# group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} -# cancel-in-progress: true +# GitHub can deliver a PR's pull_request event twice for one push when the PR's base +# branch moved in the same push, as it does for every PR above the bottom of a restacked +# stack. Keyed on the PR and its head commit, the later run cancels its identical twin, +# and a run for an earlier commit is left to finish, as before (a group keyed on the +# branch would cancel it). Pushes to main key on the run id, so none is ever cancelled. +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-${{ github.event.pull_request.head.sha || github.sha }} + cancel-in-progress: true jobs: rngs: From 726a22651c1f3af595d285f519577f008504ca68 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Tue, 27 Oct 2026 08:33:09 -0500 Subject: [PATCH 13/19] Give the test and notebook steps a timeout A run that hangs holds its runner for GitHub's default six hours. One way to hang was seen locally: on macOS, a conda environment with a pip suitesparse-graphblas wheel loads two copies of libomp (the workflow sets KMP_DUPLICATE_LIB_OK=TRUE to allow that), and a full test run sat at 0% CPU for seven hours in SuiteSparse's OpenMP join barrier. "Unit tests" and "Unit tests (bizarro scalars)" now stop after 120 minutes (the slowest seen is 73.5 minutes, a full --runslow on macos-15-intel with the Tensor PR higher in the stack), and "Notebooks Execution check" after 30. --- .github/workflows/test_and_build.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 0f430629f..e44e182eb 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -295,6 +295,9 @@ jobs: - name: JIT diagnostics run: python scripts/jit_diagnostics.py || true - name: Unit tests + # A hang (e.g. two libomp copies deadlocking in an OpenMP barrier on macOS) would + # otherwise hold the runner for GitHub's 6-hour limit. + timeout-minutes: 120 run: | A=${{ needs.rngs.outputs.mapnumpy == 'A' || '' }} ; B=${{ needs.rngs.outputs.mapnumpy == 'B' || '' }} C=${{ needs.rngs.outputs.mapnumpy == 'C' || '' }} ; D=${{ needs.rngs.outputs.mapnumpy == 'D' || '' }} @@ -327,6 +330,7 @@ jobs: coverage run -m pytest --color=yes --randomly -v ${args} \ ${{ matrix.slowtask == 'pytest_normal' && '--runslow' || '' }} - name: Unit tests (bizarro scalars) + timeout-minutes: 120 run: | # Run tests again with Scalars being C scalars by default find graphblas -type f -name "*.py" -print0 | xargs -0 sed -i -s \ @@ -426,6 +430,7 @@ jobs: token: ${{ secrets.CODECOV_TOKEN }} - name: Notebooks Execution check if: matrix.slowtask == 'notebooks' + timeout-minutes: 30 run: | # Run notebooks only if numba is installed if python -c 'import numba' 2> /dev/null ; then From 6c46514933017a6c32c31a7511d90b603182f313 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Tue, 27 Oct 2026 15:09:36 -0500 Subject: [PATCH 14/19] Keep conda-forge's libomp out of macOS jobs that use the psg wheel The suitesparse-graphblas wheel for macOS bundles its own libomp. When conda-forge's libomp is in the same process too (KMP_DUPLICATE_LIB_OK=TRUE lets both start), the two runtimes call into each other's barrier code: a GraphBLAS parallel region (a large calloc's memset) can wait forever in its join barrier, or a worker crashes. A macOS notebooks job hung that way in CI, and locally, with the wheel, numpy 2.1, scipy 1.13 and numba 0.68, the Tensor test modules hung at the same test in every run (2 of 2), also with NUMBA_THREADING_LAYER=workqueue (1 of 1). conda-forge's libomp gets in two ways: OpenBLAS's openmp build (loaded with numpy) and Numba's default "omp" threading layer. With both replaced, by OpenBLAS's pthreads build and Numba's "workqueue" layer, the same runs finished (3 of 3, two seeds; 529 passed), and the wheel's libomp was the only OpenMP runtime the process loaded. With the pthreads build alone, Numba's omp layer still loaded conda-forge's libomp, and the run crashed in the first parallel Numba kernel. So "Update env" adds libopenblas=*=*pthreads* to the conda install and sets NUMBA_THREADING_LAYER=workqueue for later steps on macOS draws of the wheel. conda-forge has macOS pthreads builds from libopenblas 0.3.30 on only, and scipy 1.9-1.11 require an older libopenblas, so draws pinning those keep the openmp build. Dry-run solves of the install line with the pin resolve on osx-arm64 and osx-64 for Python 3.11 to 3.15 (libopenblas 0.3.34 pthreads). --- .github/workflows/test_and_build.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index e44e182eb..1477b750e 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -108,8 +108,10 @@ jobs: # - os: ubuntu-24.04-arm # Linux aarch64 # slowtask: notebooks env: - # Wheels on OS X come with an OpenMP that conflicts with OpenMP from conda-forge. - # Setting this is a workaround. + # Wheels on OS X come with an OpenMP that conflicts with OpenMP from conda-forge: + # libomp aborts when a second copy starts in the process, unless this is set. Two + # copies can still deadlock or crash in an OpenMP barrier, so "Update env" keeps + # conda-forge's libomp out of macOS wheel draws where it can. KMP_DUPLICATE_LIB_OK: ${{ contains(matrix.os, 'macos') && 'TRUE' || 'FALSE' }} steps: - name: Checkout @@ -193,6 +195,22 @@ jobs: graphblas="graphblas>=7.4" fi + # A psg wheel on macOS bundles its own libomp, and with conda-forge's libomp in the + # same process too, a GraphBLAS parallel region can hang or crash (see + # KMP_DUPLICATE_LIB_OK). conda-forge's libomp comes in with OpenBLAS's openmp build + # and with Numba's default "omp" threading layer, so take OpenBLAS's pthreads build + # and Numba's "workqueue" layer instead. conda-forge has pthreads builds for macOS + # from libopenblas 0.3.30 on only, which scipy 1.9-1.11 do not accept, so those + # draws keep the openmp build. + blas="" + if [[ $SOURCETYPE == "wheel" && ${{ contains(matrix.os, 'macos') }} == true ]] ; then + case $spver in + =1.9 | =1.10 | =1.11) ;; + *) blas="libopenblas=*=*pthreads*" ;; + esac + echo "NUMBA_THREADING_LAYER=workqueue" >> "$GITHUB_ENV" + fi + # Decide whether to install numba and sparse. # Random skip: pypy always, 20% on Windows, 40% on other OSes (unless notebooks). awkward=awkward${akver} @@ -257,7 +275,7 @@ jobs: ${{ matrix.slowtask == 'pytest_bizarro' && 'black' || '' }} \ ${{ matrix.slowtask == 'notebooks' && '"matplotlib${mplver}" nbconvert jupyter "ipython>=7" drawsvg' || '' }} \ ${{ steps.sourcetype.outputs.selected == 'upstream' && 'cython' || '' }} \ - ${graphblas} \ + ${graphblas} ${blas:+"$blas"} \ ${interpreter} \ ${{ matrix.os == 'windows-latest' && 'cmake' || 'm4' }} \ # ${{ matrix.os != 'windows-latest' && 'pytest-forked' || '' }} # to investigate crashes From 4ec85610eead0e6079b1cc9412d599bcbfe9cef1 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Tue, 27 Oct 2026 23:40:02 -0500 Subject: [PATCH 15/19] Give the other test steps a 30-minute timeout, as the notebooks step has "Miscellaneous tests", "More tests for coverage" and "JIT tests for coverage" run graphblas too, so the hang the unit-test timeouts guard against (two libomp copies on macOS) could hold their runner for the 6-hour limit as well. In the two green CI runs of 50-tensor on 2026-10-27, "Miscellaneous tests" took 0.1 to 0.5 minutes and "More tests for coverage" 1.6 to 2.1. "JIT tests for coverage" has not run in CI yet; with a cold JIT cache at the top of the stack it took 62 seconds locally, compiling 227 kernels. --- .github/workflows/test_and_build.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 1477b750e..2528926f5 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -389,6 +389,7 @@ jobs: git checkout . # Undo changes to scalar default - name: Miscellaneous tests if: matrix.slowtask == 'pytest_normal' + timeout-minutes: 30 run: | # Test (and cover) automatic initialization coverage run -a graphblas/tests/test_auto_init.py @@ -412,6 +413,7 @@ jobs: # coverage run -a -m pytest --color=yes -x --no-mapnumpy -k test_npmonoid graphblas/tests/test_numpyops.py --runslow - name: More tests for coverage if: matrix.slowtask == 'notebooks' && matrix.os == 'windows-latest' + timeout-minutes: 30 run: | # We use 'notebooks' slow task b/c it should have numba installed coverage run -a -m pytest --color=yes --runslow --no-mapnumpy -p no:randomly -v -k 'test_commutes or test_bool_doesnt_get_too_large or test_npbinary or test_npmonoid or test_npsemiring' @@ -419,6 +421,7 @@ jobs: coverage run -a -m pytest --color=yes -x --no-mapnumpy --runslow -k test_binaryop_attributes_numpy graphblas/tests/test_op.py - name: JIT tests for coverage if: matrix.slowtask == 'notebooks' && matrix.os != 'windows-latest' + timeout-minutes: 30 run: | # Most of test_ssjit.py needs the suitesparse backend, --runslow, Numba and a # GraphBLAS that can JIT-compile, all in one pytest run. In one run in four or five, From 5239a19e0f3867e497fc23bb57a5e1555349757f Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Wed, 28 Oct 2026 19:07:46 -0500 Subject: [PATCH 16/19] Add python-suitesparse-graphblas 10.5.2.0 to CI's version draws psg 10.5.2.0 is on PyPI and conda-forge. PyPI has cp311 to cp315 wheels for every platform CI runs on (manylinux x86_64 and aarch64, macOS arm64 and x86_64, win_amd64), and the sdist. conda-forge has builds for py311 to py315 on linux-64, linux-aarch64, osx-64, osx-arm64 and win-64, all depending on graphblas >=10.5.2,<10.6, which conda-forge has on all five, so a "source" draw's graphblas pin solves too. The builds need numpy >=1.24 (py3.11) or >=1.25 (py3.12 on), which the numpy 2.x pool this list feeds always meets. No Python floor moves: 10.5.2.0 builds for every Python 3.11 to 3.15 on every source, so PSG_MIN_PYTHON stays as it is. --- scripts/check_versions.sh | 2 +- scripts/ci_pick_versions.py | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/scripts/check_versions.sh b/scripts/check_versions.sh index 05a17b12b..a7e352363 100755 --- a/scripts/check_versions.sh +++ b/scripts/check_versions.sh @@ -25,5 +25,5 @@ conda search 'pyyaml[channel=conda-forge]>=6.0' # matplotlib is installed unpinned for the notebooks job; ci_pick_versions.py caps it # below 3.11 when the numpy pick predates matplotlib's numpy >=1.25 runtime floor. conda search 'matplotlib-base[channel=conda-forge]>=3.11' -conda search 'python-suitesparse-graphblas[channel=conda-forge]>=10.5.1' +conda search 'python-suitesparse-graphblas[channel=conda-forge]>=10.5.2' # conda search 'python[channel=conda-forge]>=3.11 *pypy*' diff --git a/scripts/ci_pick_versions.py b/scripts/ci_pick_versions.py index 2d7c296e8..34f76e9ee 100755 --- a/scripts/ci_pick_versions.py +++ b/scripts/ci_pick_versions.py @@ -216,6 +216,7 @@ "10.4.1.0", "10.5.0.0", "10.5.1.1", + "10.5.2.0", ] # Oldest psg release that installs on a given Python, by where psg comes from. @@ -231,7 +232,7 @@ # py3.15 starts at 10.5.1.1 everywhere: it is psg's first release with py315 builds on # conda-forge and cp315 wheels on PyPI, and its sdist is the first from after psg added # Python 3.15 support. -# MAINT: 2026-10-18 checked conda-forge build strings and PyPI wheel tags for psg 9-10 +# MAINT: 2026-10-28 checked conda-forge build strings and PyPI wheel tags for psg 9-10 PSG_MIN_PYTHON = { "3.14": {"conda-forge": "10.0.1.1", "wheel": "10.3.1.0", "source": "10.3.1.0"}, "3.15": {"conda-forge": "10.5.1.1", "wheel": "10.5.1.1", "source": "10.5.1.1"}, From f6275f2c23f3f344ee6f75140a7f466333fd06c6 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Wed, 28 Oct 2026 19:17:06 -0500 Subject: [PATCH 17/19] Give a pinned psg the GraphBLAS it was built for on conda-forge and source draws python-suitesparse-graphblas X.Y.Z.W wraps SuiteSparse:GraphBLAS X.Y.Z, and psg's own test_version_tracks_libgraphblas asserts that the two match. conda-forge's psg builds accept any graphblas >=X.Y.Z,=X.Y.Z,=X.Y.Z on conda-forge, and graphblas X.Y.Z is there for linux-64, linux-aarch64, osx-64, osx-arm64 and win-64. Unpinned psg, upstream and wheel draws are unchanged. Checked with dry-run solves of CI's spec list, 17 of 17 landing on graphblas X.Y.Z: conda-forge draws of psg 10.5.1.1 on py3.15 on all five platforms, of an old psg on py3.11 on each (7.4.0 on linux-64 and osx-arm64, 7.4.3.2 on linux-aarch64, 8.0.2.1 on osx-64, 7.4.1 on win-64), of 8.2.1.0, 9.3.1.0, 9.4.5.0 and 10.5.2.0, and source draws of 9.4.5.0, 10.3.1.0 and 10.5.1.1. Without the pin, a py3.13 osx-arm64 conda-forge draw of 10.5.1.1 solves to graphblas 10.5.2. --- .github/workflows/test_and_build.yml | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/.github/workflows/test_and_build.yml b/.github/workflows/test_and_build.yml index 2528926f5..dadb23e36 100644 --- a/.github/workflows/test_and_build.yml +++ b/.github/workflows/test_and_build.yml @@ -183,14 +183,18 @@ jobs: # step's shell variables no longer exist, so psgver goes there as an output. echo "psgver=$psgver" >> "$GITHUB_OUTPUT" - # Wheels bundle their own GraphBLAS. A source draw compiles psg's sdist against - # conda-forge's, so give it the one conda-forge pairs with that psg release: - # python-suitesparse-graphblas X.Y.Z.W depends on graphblas >=X.Y.Z,=X.Y.Z, Date: Tue, 3 Nov 2026 04:00:16 -0800 Subject: [PATCH 18/19] Bump pypa/gh-action-pypi-publish to v1.14.3 for GHSA-5xjx-3w67-w9x9 v1.14.2 and earlier trust a certificate authority that Sigstore has revoked, which Trusted Publishing and digital attestations rely on. zizmor's known-vulnerable-actions audit fails on the old pin. --- .github/workflows/publish_pypi.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/publish_pypi.yml b/.github/workflows/publish_pypi.yml index 3370664f9..fd2f3af10 100644 --- a/.github/workflows/publish_pypi.yml +++ b/.github/workflows/publish_pypi.yml @@ -54,4 +54,4 @@ jobs: name: releases path: dist - name: Publish to PyPI - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3 From 3f364118e1a81f1a10c389ac413254661c188d86 Mon Sep 17 00:00:00 2001 From: Erik Welch Date: Tue, 3 Nov 2026 07:13:50 -0800 Subject: [PATCH 19/19] Update the pre-commit hooks ruff 0.17.3, flake8 7.5.0 (with flake8-bugbear 26.10.27), zizmor 1.32.0, validate-pyproject 0.27 and check-jsonschema 0.39.0. The isort and pyroma updates pre-commit autoupdate offers are betas (10.0.0b2, 5.1b1), so they stay where they are. pre-commit run --all-files passes. --- .pre-commit-config.yaml | 16 ++++++++-------- scripts/check_versions.sh | 2 +- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index dc1695667..8515f7f41 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -43,7 +43,7 @@ repos: - id: name-tests-test args: ["--pytest-test-first"] - repo: https://github.com/abravalheri/validate-pyproject - rev: "0.26" + rev: v0.27 hooks: - id: validate-pyproject name: Validate pyproject.toml @@ -76,21 +76,21 @@ repos: - id: black - id: black-jupyter - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.17.1 + rev: v0.17.3 hooks: - id: ruff-check args: [--fix-only, --show-fixes] # Let's keep `flake8` even though `ruff` does much of the same. # `flake8-bugbear` and `flake8-simplify` have caught things missed by `ruff`. - repo: https://github.com/PyCQA/flake8 - rev: 7.4.1 + rev: 7.5.0 hooks: - id: flake8 args: ["--config=.flake8"] additional_dependencies: &flake8_dependencies # These versions need updated manually - - flake8==7.4.1 - - flake8-bugbear==26.9.30 + - flake8==7.5.0 + - flake8-bugbear==26.10.27 - flake8-simplify==0.31.1 - repo: https://github.com/codespell-project/codespell rev: v2.4.3 @@ -101,7 +101,7 @@ repos: - tomli; python_version<'3.11' files: ^(graphblas|docs)/ - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.17.1 + rev: v0.17.3 hooks: - id: ruff-check # - id: ruff-format # we prefer black; uncomment to compare with ruff's formatter @@ -149,11 +149,11 @@ repos: hooks: - id: actionlint - repo: https://github.com/zizmorcore/zizmor-pre-commit - rev: v1.31.1 + rev: v1.32.0 hooks: - id: zizmor - repo: https://github.com/python-jsonschema/check-jsonschema - rev: 0.38.2 + rev: 0.39.0 hooks: - id: check-dependabot - id: check-github-workflows diff --git a/scripts/check_versions.sh b/scripts/check_versions.sh index a7e352363..e99cb7ca6 100755 --- a/scripts/check_versions.sh +++ b/scripts/check_versions.sh @@ -10,7 +10,7 @@ # scripts/audit_pypi_versions.py and scripts/audit_conda_versions.py ask the same # questions automatically, and also check the pools and tables in ci_pick_versions.py # against what is published. The "Audit dependency versions" workflow runs both weekly. -conda search 'flake8-bugbear[channel=conda-forge]>=26.9.30' +conda search 'flake8-bugbear[channel=conda-forge]>=26.10.27' conda search 'flake8-simplify[channel=conda-forge]>=0.31.1' conda search 'numpy[channel=conda-forge]>=2.5' # ci_pick_versions.py caps an unpinned pandas below 3.1 for now (see its MAINT note).