CUDA Templates for Linear Algebra Subroutines

Fast and memory-efficient exact attention

FlashInfer: Kernel Library for LLM Serving

A high-throughput and memory-efficient inference and serving engine for LLMs

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

Optimized primitives for collective multi-GPU communication

DeepEP: an efficient expert-parallel communication library

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

A highly optimized inference acceleration engine for Llama and its variants.

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

cuda code benchmark

📚 Computer Science Books 计算机技术类书籍 PDF

Python functions for reading kaldi data formats. Useful for rapid prototyping with python.

中文命名实体识别，实体抽取，tensorflow，pytorch，BiLSTM+CRF

Keras Interface for Kaldi ASR