Skip to content

Benchmark Dispatch #310

Benchmark Dispatch

Benchmark Dispatch #310

Manually triggered December 23, 2025 08:03
Status Failure
Total duration 1d 0h 0m 9s
Artifacts 15

benchmark_dispatch.yml

on: workflow_dispatch
gen-matrix-b200  /  gen
3s
gen-matrix-b200 / gen
gen-matrix-h100  /  gen
3s
gen-matrix-h100 / gen
gen-matrix-mi325x  /  gen
2s
gen-matrix-mi325x / gen
Matrix: run-b200
Matrix: run-h100
Matrix: run-mi325x
Fit to window
Zoom out
Zoom in

Annotations

71 errors
run-b200 (grouped_gemm) / benchmark-cu130-grouped_gemm-py3.12-b200
Process completed with exit code 1.
run-b200 (grouped_gemm) / benchmark-cu130-grouped_gemm-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (grouped_gemm) / benchmark-cu130-grouped_gemm-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (jsd) / benchmark-cu130-jsd-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (jsd) / benchmark-cu130-jsd-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (grouped_gemm) / benchmark-cu128-grouped_gemm-py3.12-h100
Process completed with exit code 1.
run-h100 (grouped_gemm) / benchmark-cu128-grouped_gemm-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (grouped_gemm) / benchmark-cu128-grouped_gemm-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (jsd) / benchmark-cu128-jsd-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (jsd) / benchmark-cu128-jsd-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (rms_norm) / benchmark-cu130-rms_norm-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (rms_norm) / benchmark-cu130-rms_norm-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (int4_gemm) / benchmark-cu130-int4_gemm-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (int4_gemm) / benchmark-cu130-int4_gemm-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (kl_div) / benchmark-cu128-kl_div-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (kl_div) / benchmark-cu128-kl_div-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (kl_div) / benchmark-cu130-kl_div-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (kl_div) / benchmark-cu130-kl_div-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (cross_entropy) / benchmark-cu130-cross_entropy-py3.12-b200
Process completed with exit code 1.
run-b200 (cross_entropy) / benchmark-cu130-cross_entropy-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (cross_entropy) / benchmark-cu130-cross_entropy-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (cross_entropy) / benchmark-cu128-cross_entropy-py3.12-h100
Process completed with exit code 1.
run-h100 (cross_entropy) / benchmark-cu128-cross_entropy-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (cross_entropy) / benchmark-cu128-cross_entropy-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (rms_norm) / benchmark-cu128-rms_norm-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (rms_norm) / benchmark-cu128-rms_norm-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (gemm) / benchmark-cu130-gemm-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (gemm) / benchmark-cu130-gemm-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (layer_norm) / benchmark-cu130-layer_norm-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (layer_norm) / benchmark-cu130-layer_norm-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (rms_norm-bwd) / benchmark-cu130-rms_norm-bwd-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (rms_norm-bwd) / benchmark-cu130-rms_norm-bwd-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (softmax) / benchmark-cu130-softmax-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (softmax) / benchmark-cu130-softmax-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (gemm) / benchmark-cu128-gemm-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (gemm) / benchmark-cu128-gemm-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (flash_attention) / benchmark-cu130-flash_attention-py3.12-b200
Process completed with exit code 1.
run-b200 (flash_attention) / benchmark-cu130-flash_attention-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (flash_attention) / benchmark-cu130-flash_attention-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (rms_norm-bwd) / benchmark-cu128-rms_norm-bwd-py3.12-h100
Process completed with exit code 1.
run-h100 (rms_norm-bwd) / benchmark-cu128-rms_norm-bwd-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (rms_norm-bwd) / benchmark-cu128-rms_norm-bwd-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (welford) / benchmark-cu130-welford-py3.12-b200
Process completed with exit code 1.
run-b200 (welford) / benchmark-cu130-welford-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (welford) / benchmark-cu130-welford-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (layer_norm) / benchmark-cu128-layer_norm-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (layer_norm) / benchmark-cu128-layer_norm-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-b200 (layer_norm-bwd) / benchmark-cu130-layer_norm-bwd-py3.12-b200
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-b200 (layer_norm-bwd) / benchmark-cu130-layer_norm-bwd-py3.12-b200
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (welford) / benchmark-cu128-welford-py3.12-h100
Process completed with exit code 1.
run-h100 (welford) / benchmark-cu128-welford-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (welford) / benchmark-cu128-welford-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (flash_attention) / benchmark-cu128-flash_attention-py3.12-h100
Process completed with exit code 1.
run-h100 (flash_attention) / benchmark-cu128-flash_attention-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (flash_attention) / benchmark-cu128-flash_attention-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (softmax) / benchmark-cu128-softmax-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (softmax) / benchmark-cu128-softmax-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (int4_gemm) / benchmark-cu128-int4_gemm-py3.12-h100
The job has exceeded the maximum execution time of 6h0m0s
run-h100 (int4_gemm) / benchmark-cu128-int4_gemm-py3.12-h100
The operation was canceled.
run-h100 (int4_gemm) / benchmark-cu128-int4_gemm-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (int4_gemm) / benchmark-cu128-int4_gemm-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-h100 (layer_norm-bwd) / benchmark-cu128-layer_norm-bwd-py3.12-h100
The job has exceeded the maximum execution time of 6h0m0s
run-h100 (layer_norm-bwd) / benchmark-cu128-layer_norm-bwd-py3.12-h100
The operation was canceled.
run-h100 (layer_norm-bwd) / benchmark-cu128-layer_norm-bwd-py3.12-h100
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
run-h100 (layer_norm-bwd) / benchmark-cu128-layer_norm-bwd-py3.12-h100
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
run-mi325x (welford,rms_norm-bwd) / benchmark-rocm6.4-welford,rms_norm-bwd-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s
run-mi325x (layer_norm,gemm) / benchmark-rocm6.4-layer_norm,gemm-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s
run-mi325x (jsd,rms_norm) / benchmark-rocm6.4-jsd,rms_norm-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s
run-mi325x (int4_gemm,flash_attention) / benchmark-rocm6.4-int4_gemm,flash_attention-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s
run-mi325x (softmax,layer_norm-bwd,grouped_gemm) / benchmark-rocm6.4-softmax,layer_norm-bwd,grouped_gemm-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s
run-mi325x (kl_div,cross_entropy) / benchmark-rocm6.4-kl_div,cross_entropy-py3.12-mi325x
The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

Artifacts

Produced during runtime
Name Size Digest
benchmark-results-b200-gemm
681 Bytes
sha256:b03d737235c74225b6b7f0e9501f10d2680abf29b7536ace058f459eb50f11e5
benchmark-results-b200-int4_gemm
561 Bytes
sha256:7359cf4c1704444217bc1315b0539d360a8b4b2c35f294d13a313e62b9c21498
benchmark-results-b200-jsd
150 Bytes
sha256:73d465690930965c637db425625e539673c4169c90785ce7ef71166e163cdd39
benchmark-results-b200-kl_div
618 Bytes
sha256:0611713de75361887c3c3f47d642f8b224552bdc48c206d24b7364947dbebb53
benchmark-results-b200-layer_norm
1.06 KB
sha256:29e1c50b8788f28a131e4855954ceacddf8dcbaa62fd0ad88cbe48c9219d6b50
benchmark-results-b200-layer_norm-bwd
756 Bytes
sha256:08ecb9dbed012124054bf8230f045e9a44213cbb613954ef7a1835afa4696d0d
benchmark-results-b200-rms_norm
619 Bytes
sha256:9428a95157acbcf28eed0d77997dc2eeefecdb55eb5ae7a75e04bfcf46350dc7
benchmark-results-b200-rms_norm-bwd
592 Bytes
sha256:ffef212233673dd363f4383e58f33af6742c1113fd981f8d6a0e3d2d5ed549d9
benchmark-results-b200-softmax
1.06 KB
sha256:24db4c5fbd38822b129dddf0a852baf0cb7594aa7e5272aeb92d4e88fe7ffe8e
benchmark-results-h100-gemm
682 Bytes
sha256:04f5eb23a882c55e13f7d94c215244bc4c0365fd1f86726d1f35b6a6086933f7
benchmark-results-h100-jsd
150 Bytes
sha256:7f3ca9dd61f34e9ee4bba451347d442126e75a5b4ac5ec04ca9906e79f0f30fb
benchmark-results-h100-kl_div
616 Bytes
sha256:64b867de84c11fe329269714f56309e1c362b534cbae01923a76af65aa7b0726
benchmark-results-h100-layer_norm
1.05 KB
sha256:fc5d2b5a508d905181859352f582cde5f2f019ae05f2340578bb7dd3d9950cdc
benchmark-results-h100-rms_norm
617 Bytes
sha256:6530a743eef26170f979693e8d40099692157ace036e2dbd14400335cf04aaf1
benchmark-results-h100-softmax
1.05 KB
sha256:b7fb6e96f7e3ba96391a4292233211685dc46e22b0728189b45f962bb8ab119e