Lists (1)
Stars
verl: Volcano Engine Reinforcement Learning for LLMs
EvaByte: Efficient Byte-level Language Models at Scale
If your linear layer is secretly fast weight memory, why not model fast weight memory updates as optimizers?
Understand and test language model architectures on synthetic tasks.
Ongoing research training transformer models at scale
The simplest, fastest repository for training/finetuning medium-sized GPTs.
Scaling Diffusion Transformers with Mixture of Experts
Scalable toolkit for efficient model reinforcement
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI
🔥 A minimal training framework for scaling FLA models
🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
NanoGPT (124M) in 3 minutes
Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, 20+ clouds, or on-prem).
Minimal implementation of scalable rectified flow transformers, based on SD3's approach
Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
Reference implementations of MLPerf® training benchmarks
(CVPR 2025) From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Accelerator (NeurIPS 2024)
Modeling, training, eval, and inference code for OLMo
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training