🤗A PyTorch-native Inference Engine with Hybrid Cache Acceleration and Parallelism for DiTs: Z-Image, FLUX2, Qwen-Image, etc.

A high-throughput and memory-efficient inference and serving engine for LLMs

SGLang is a fast serving framework for large language models and vision language models.

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

FlashInfer: Kernel Library for LLM Serving

My learning notes/codes for ML SYS.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Machine Learning Engineering Open Book

HunyuanVideo: A Systematic Framework For Large Video Generation Model

PyTorch native quantization and sparsity for training and inference

Fast and memory-efficient exact attention

TiledCUDA is a highly efficient kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

nndeploy是一款模型端到端部署框架。以多端推理以及基于有向无环图模型部署为内核，致力为用户提供跨平台、简单易用、高性能的模型部署体验。

A CPU+GPU Profiling library that provides access to timeline traces and hardware performance counters.