Training library for Megatron-based models

slime is an LLM post-training framework for RL Scaling.

Pretrained universal neural network potential for charge-informed atomistic modeling https://chgnet.lbl.gov

NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the effective training time by minimizing the downtime due to fa…

wheels for TransformerEngine

A PyTorch native platform for training generative AI models

Ongoing research training transformer models at scale

Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.

Parallel S3 and local filesystem execution tool.

A high-throughput and memory-efficient inference and serving engine for LLMs

Scalable toolkit for efficient model reinforcement

verl: Volcano Engine Reinforcement Learning for LLMs

Official Repo for Open-Reasoner-Zero

Minimalistic 4D-parallelism distributed training framework for education purpose

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

The official Python client for the Huggingface Hub.

LeanRL is a fork of CleanRL, where selected PyTorch scripts optimized for performance using compile and cudagraphs.

Train transformer language models with reinforcement learning.

Public repo for HF blog posts