FlashMLA: Efficient MLA decoding kernels

讨贼王云鹤檄文

📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA/Tensor Cores Kernels, HGEMM, FA-2 MMA.🎉

🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!

SGLang is yet another fast serving framework for large language models and vision language models.

Implement Flash Attention using Cute.

Fastest kernels written from scratch

Material for gpu-mode lectures

Applied AI experiments and examples for PyTorch

how to optimize some algorithm in cuda.

FlagGems is an operator library for large language models implemented in Triton Language.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

Development repository for the Triton language and compiler

Fast and memory-efficient exact attention

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

collection of benchmarks to measure basic GPU capabilities

Llama中文社区，Llama3在线体验和微调模型已开放，实时汇总最新Llama3学习资料，已将所有代码更新适配Llama3，构建最好的中文Llama大模型，完全开源可商用

Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory

Scripts for fine-tuning Llama2 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization & question answering. S…