-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Open
Labels
enhancementNew feature or requestNew feature or request
Description
https://github.com/allenai/OLMoE is a very good MOE structure, with 7B parameter and 1B activate parameter. It is very suitable for academic research. Whether the litgpt team considers to support this model?
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request