SGLang is a fast serving framework for large language models and vision language models.

slime is a LLM post-training framework aiming at scaling RL.

Sentry SDK for Dart and Flutter

FlashInfer: Kernel Library for LLM Serving

DeepEP: an efficient expert-parallel communication library

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory…

Development repository for the Triton language and compiler

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

LongBench v2 and LongBench (ACL 25'&24')

CUDA Templates for Linear Algebra Subroutines

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

A project to improve skills of large language models

Kimina Lean server

veRL: Volcano Engine Reinforcement Learning for LLM

bridges the log crate and indicatif to stop the progress bars and log lines from mixing up

Debug print operator for cudagraph debugging

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.