Skip to content

[cuDNN][CUDA] Allow Flash attention or cuDNN attention as default in fused_sdp_choice test #159582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[cuDNN][CUDA] Allow Flash attention or cuDNN attention as default in fused_sdp_choice test #159582

wants to merge 1 commit into from
+4 −1

Conversation

eqy
Copy link
Collaborator

@eqy eqy commented Jul 31, 2025 •
edited by pytorch-bot bot
Loading

Hopefully we'll be able to flip the default soon...

cc @csarofeen @ptrblck @xwang233 @msaroufim @jerryzh168

@eqy eqy added module: cudnn Related to torch.backends.cudnn, and CuDNN support module: cuda Related to torch.cuda, and CUDA support in general open source ciflow/trunk Trigger trunk jobs on your pull request topic: not user facing topic category module: sdpa All things related to torch.nn.functional.scaled_dot_product_attentiion labels Jul 31, 2025
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 31, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159582

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 10919d3 with merge base f89c28c (image):

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@janeyx99 janeyx99 requested a review from drisspg August 4, 2025 23:09
@janeyx99 janeyx99 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Aug 4, 2025
drisspg
drisspg reviewed Aug 5, 2025
View reviewed changes
test/test_transformers.py
# TODO we are currently disabling this by default, lets assert that this returns
# FlashAttention, we need to change when we make remove opt-in for cudnn
if type != "nested" and PLATFORM_SUPPORTS_CUDNN_ATTENTION and SM90OrLater:
self.assertEqual(torch._fused_sdp_choice(query, key, value), SDPBackend.FLASH_ATTENTION.value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we just self.assert (... in [Flash, CuDNN])

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 4, 2025

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label Oct 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@drisspg drisspg drisspg left review comments

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request module: cuda Related to torch.cuda, and CUDA support in general module: cudnn Related to torch.backends.cudnn, and CuDNN support module: sdpa All things related to torch.nn.functional.scaled_dot_product_attentiion open source Stale topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@eqy @drisspg @janeyx99