Lightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.

slime is an LLM post-training framework for RL Scaling.

Multilingual Document Layout Parsing in a Single Vision-Language Model

VeOmni: Scaling any Modality Model Training to any Accelerators with PyTorch native Training Framework

Bridge Megatron-Core to Hugging Face/Reinforcement Learning

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

SGLang is a fast serving framework for large language models and vision language models.

A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment…

An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & RFT & Dynamic Sampling & Async Agent RL)

Scalable toolkit for efficient model reinforcement

📚A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism etc.

verl: Volcano Engine Reinforcement Learning for LLMs

Awesome LLM compression research papers and tools.

主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题

A high-throughput and memory-efficient inference and serving engine for LLMs

📰 Must-read papers and blogs on Speculative Decoding ⚡️

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

[ICML'24] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

LLM inference in C/C++

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

NeMo: a framework for generative AI

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

LAVIS - A One-stop Library for Language-Vision Intelligence

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Microsoft Automatic Mixed Precision Library

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…