Lists (1)
Stars
Light Image Video Generation Inference Framework
A unified inference and post-training framework for accelerated video generation.
TurboDiffusion: 100–200× Acceleration for Video Diffusion Models
rCM: SOTA Diffusion Distillation & Few-Step Video Generation based on sCM/MeanFlow
Accelerating MoE with IO and Tile-aware Optimizations
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
🤗A PyTorch-native and Flexible Inference Engine with Hybrid Cache Acceleration and Parallelism for DiTs.
GPU programming related news and material links
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
Expert Specialization MoE Solution based on CUTLASS
Utility scripts for PyTorch (e.g. Make Perfetto show some disappearing kernels, Memory profiler that understands more low-level allocations such as NCCL, ...)
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Unleashing the Power of Reinforcement Learning for Math and Code Reasoners
A Datacenter Scale Distributed Inference Serving Framework
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
verl: Volcano Engine Reinforcement Learning for LLMs
A bidirectional pipeline parallelism algorithm for computation-communication overlap in DeepSeek V3/R1 training.
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels