[inductor][eazy] change how torch.use_deterministic_algorithms affect inductor #164905
Conversation
… inductor [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/164905
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 New Failure, 1 PendingAs of commit c9c4fe6 with merge base 50c338c ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
…thms affect inductor" Previously when torch.are_deterministic_algorithms_enabled() is True Inductor will - skip autotuning pointwise kernels - pick a fixed (and quite arbitrary) config for reduction This PR change the behavior to - for pointwise kernels, we still do autotuning - for reduction kernels, we use the recent added heuristic to pick a config cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: trunk / linux-jammy-cuda12.8-py3.10-gcc11 / test (default, 3, 5, lf.linux.g6.4xlarge.experimental.nvidia.gpu) Details for Dev Infra teamRaised by workflow job |
…thms affect inductor" Previously when torch.are_deterministic_algorithms_enabled() is True Inductor will - skip autotuning pointwise kernels - pick a fixed (and quite arbitrary) config for reduction This PR change the behavior to - for pointwise kernels, we still do autotuning - for reduction kernels, we use the recent added heuristic to pick a config cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 4 checks: inductor / inductor-test / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu), trunk / macos-py3-arm64 / test (default, 2, 3, macos-m1-stable), trunk / macos-py3-arm64 / test (default, 1, 3, macos-m1-stable), trunk / macos-py3-arm64 / test (default, 3, 3, macos-m1-stable) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
@pytorchbot revert -m 'Sorry for reverting your PR but there seems to be some failed vLLM failures coming out of this' -c nosignal I'm not sure if the failure is related, so let's revert this first to see if it fixes trunk. I will monitor and reland this if trunk remain broken |
|
@pytorchbot successfully started a revert job. Check the current status here. |
|
@shunting314 your PR has been successfully reverted. |
…s affect inductor (#164905)" This reverts commit 344e636. Reverted #164905 on behalf of https://github.com/huydhn due to Sorry for reverting your PR but there seems to be some failed vLLM failures coming out of this ([comment](#164905 (comment)))
|
@huydhn any label I can add to run those vllm tests in CI before landing the PR? |
…thms affect inductor" Previously when torch.are_deterministic_algorithms_enabled() is True Inductor will - skip autotuning pointwise kernels - pick a fixed (and quite arbitrary) config for reduction This PR change the behavior to - for pointwise kernels, we still do autotuning - for reduction kernels, we use the recent added heuristic to pick a config cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]
Stack from ghstack (oldest at bottom):
Previously when torch.are_deterministic_algorithms_enabled() is True Inductor will
This PR change the behavior to
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben