🐛 Describe the bug

Random ATen operations that use MPS backend can fail with Error: total bytes of NDArray > 2**32 or other such errors that expect either tensor size to be less than 4Gb, or total number of elements to be indexable by 32-bit index.

This is umbrella issue to track those (should be searchable by module: mps+ module: 64-bit)

Need to figure out some tooling to detect broken ops and how to run tests, as allocating large tensors on machines we have right now simply not going to work

Example of existing issues:

• MPS Error: NDArray > 2^32 bytes in scaled_dot_product_attention #149261
• On the Apple M1 GPU, torch.where() finds tensor entries in the wrong place  #140570
• MPS torch.where() is giving objectively incorrect results, leading to critical calculation errors #122916
• MPS MatMul (even 2d) broken #116769
• MPS MatMul (even 2d) broken #116769
• MPSNDArray 限制了单个 NDArray 的内存大小上限为 4GB #143859
• [MPS] "Can't be indexed using 32-bit iterator" error as of 20250430 nightly cpu MacOS build #154828

Versions

CI

cc @kulinseth @albanD @DenisVieriu97 @jhavukainen