[BetterTransformer] Add MobileBERT support for BetterTransformer
#506
Conversation
There was a problem hiding this comment.
Thanks a lot for your great work @raghavanone and for start adding BetterTransformer support for MobileBert!
I left few comments, several points needs to be addressed before merging. Starting from the naming of the new class. We prefer to call the new class as xxxLayerBetterTransformer to be able to correctly trace whether the conversion has been successfull or not.
Also could you make sure this implementation is tested by adding hf-internal-testing/tiny-random-MobileBertModel here
Let us know if you face into any issue here!
MobileBERT support for BetterTransformer
|
hey @raghavanone ! |
Done |
|
|
||
| self.validate_bettertransformer() | ||
|
|
||
| def forward(self, hidden_states, attention_mask, *_): |
There was a problem hiding this comment.
What is the purpose of *_ in the end of the arguments?
There was a problem hiding this comment.
I was not sure, I saw *_ in every other forward call definition.
There was a problem hiding this comment.
Alright, maybe we should also add **__ or something to handle potential keyword arguments?
@younesbelkada
There was a problem hiding this comment.
I saw both some forward def having both *_ and **_ . We should possibly add more information about it in the guide published.
There was a problem hiding this comment.
Yes basically it is to "handle" potential arguments that were passed to the original forward function, by allowing this forward function to get them, but ignoring them right after.
There was a problem hiding this comment.
great remark, the guide will be updated ;)
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
|
The test is failing for few reasons, firstly MobileBERT uses a special mechanism called also, it would be cool to add a sanity check on the |
|
@younesbelkada let me know what needs to be done to merge this . |
|
There are conflicts, you need to fetch on the main branch and merge / rebase on this branch. |
|
This PR has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs. |
What does this PR do?
Adds mobilebertlayer