MNT: `sparse` import refactor #695

mtsokol · 2024-06-07T13:04:33Z

This PR refactors sparse importing mechanism. It enforces backend selection at the import time and allows from sparse import COO etc. imports.

github-actions · 2024-06-07T13:06:02Z

Test Results

44 tests - 6 063 31 ✅ - 5 954 37s ⏱️ - 10m 3s
1 suites ± 0 13 💤 - 109
1 files ± 0 0 ❌ ± 0

Results for commit 2dab3d2. ± Comparison against base commit 38d1a67.

This pull request removes 6107 and adds 44 tests. Note that renamed tests count towards both.

sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order0]
sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order1]
sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order2]
sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order0]
sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order1]
sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order2]
sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order0]
sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order1]
sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order2]
sparse.numba_backend.tests.test_array_function ‑ test_binary[tensordot-arg_order0]
…

sparse.tests.test_backends ‑ test_asarray[coo-C]
sparse.tests.test_backends ‑ test_asarray[coo-F]
sparse.tests.test_backends ‑ test_asarray[csc-F]
sparse.tests.test_backends ‑ test_asarray[csr-C]
sparse.tests.test_backends ‑ test_backends
sparse.tests.test_backends ‑ test_finch_lazy_backend
sparse.tests.test_backends ‑ test_scikit_learn_dispatch[csc_matrix-csc-F]
sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[coo_matrix-coo-F]
sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[csc_matrix-csc-F]
sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[csr_matrix-csr-C]
…

This pull request removes 122 skipped tests and adds 13 skipped tests. Note that renamed tests count towards both.

sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f4-axis4-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f8-axis4-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-0-mean-kwargs1]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-0-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-1-mean-kwargs1]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-1-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-axis4-mean-kwargs1]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-axis4-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-0-mean-kwargs1]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-0-sum-kwargs0]
…

sparse.tests.test_backends ‑ test_scikit_learn_dispatch[csc_matrix-csc-F]
sparse.tests.test_backends ‑ test_scipy_eigs[coo-C]
sparse.tests.test_backends ‑ test_scipy_eigs[coo-F]
sparse.tests.test_backends ‑ test_scipy_eigs[csc-F]
sparse.tests.test_backends ‑ test_scipy_eigs[csr-C]
sparse.tests.test_backends ‑ test_scipy_lsqr[coo-C]
sparse.tests.test_backends ‑ test_scipy_lsqr[coo-F]
sparse.tests.test_backends ‑ test_scipy_lsqr[csc-F]
sparse.tests.test_backends ‑ test_scipy_lsqr[csr-C]
sparse.tests.test_backends ‑ test_scipy_norm[coo-C]
…

♻️ This comment has been updated with latest results.

benchmarks/benchmark_backends.py

benchmarks/utils.py

hameerabbasi

Maybe we can run the test suite twice, once for Finch and once for Numba in CI, without doing the importlib.reload(...) magic? And provide a ./test.sh script?

mtsokol · 2024-06-10T12:44:54Z

Maybe we can run the test suite twice, once for Finch and once for Numba in CI, without doing the importlib.reload(...) magic? And provide a ./test.sh script?

Done! I run tests twice in the CI with different backends:

- name: Run tests
  run: |
    SPARSE_BACKEND=Numba pytest --pyargs sparse
    SPARSE_BACKEND=Finch pytest --pyargs sparse/tests

For Numba we run all tests. For Finch only the common ones.

hameerabbasi

This LGTM! Thanks so much for working on this, @mtsokol!

MNT: sparse import refactor

e2215b1

mtsokol added the maintenance label Jun 7, 2024

mtsokol requested a review from hameerabbasi June 7, 2024 13:04

mtsokol self-assigned this Jun 7, 2024

mtsokol mentioned this pull request Jun 7, 2024

Support direct Numba imports #689

Closed

mtsokol added 2 commits June 7, 2024 15:18

Add missing imports

4c43fac

Refactor sparse/__init__.py

6a95fa5

mtsokol force-pushed the import-refactor branch 7 times, most recently from 4aec17f to 491af3a Compare June 10, 2024 10:56

mtsokol mentioned this pull request Jun 10, 2024

Move to Array API version 2023.12. #696

Closed