Search results

Request: Add Flash Attention 2.0 Support for ViTMAEForPreTraining

Feature requestRequest for a new featureRequest for a new feature

Flash Attention

Good Second IssueIssues that are more difficult to do than "Good First" issues - give it a try if you want!Issues that are more difficult to do than "Good First" issues - give it a try if you want!

Vision

Status: Open.

#36527 In huggingface/transformers;

· noelEOS opened on Mar 4, 2025

Flex attention support with arbitrary 4d mask for LlamaModel

Feature requestRequest for a new featureRequest for a new feature

Flash Attention

Status: Open.

#33898 In huggingface/transformers;

· alex-hh opened on Oct 2, 2024

Support context parallel training with ring-flash-attention

Feature requestRequest for a new featureRequest for a new feature

Flash Attention

Good Difficult Issue

Status: Open.

#33467 In huggingface/transformers;

· zhuzilin opened on Sep 13, 2024

Flash Attention Support for Blip2ForConditionalGeneration

Feature requestRequest for a new featureRequest for a new feature

Flash Attention

Status: Open.

#30360 In huggingface/transformers;

· soumya1729 opened on Apr 20, 2024

Request for Flash Attention 2.0 Support in GPNRoFormerForMaskedLM

Feature requestRequest for a new featureRequest for a new feature

Flash Attention

Status: Open.

#29060 In huggingface/transformers;

· YBoulaimen opened on Feb 16, 2024

[Flash Attention 2] Performance improvement

Flash Attention

Good Difficult Issue

Status: Open.

#28160 In huggingface/transformers;

· li-plus opened on Dec 20, 2023

Add Flash Attention 2.0 for T5 Family

Flash Attention

New model

Status: Open.

#27441 In huggingface/transformers;

· jkswin opened on Nov 10, 2023