DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit

TVM FFI

Checkpoint-engine is a simple middleware to update model weights in LLM inference engines

NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmer…

Ultra and Unified CCL

VeOmni: Scaling any Modality Model Training to any Accelerators with PyTorch native Training Framework

gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

Triton-based Symmetric Memory operators and examples

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

Maple Mono: Open source monospace font with round corner, ligatures and Nerd-Font for IDE and terminal, fine-grained customization options. 带连字和控制台图标的圆角等宽字体，中英文宽度完美2:1，细粒度的自定义选项

Nano vLLM

TritonParse is a tool designed to help developers analyze and debug Triton kernels by visualizing the compilation process and source code mappings.

CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge techniques in sparse architecture, speculative sampling and qua…

A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.

A course of LLM inference serving on Apple Silicon for systems engineers.

kernels, of the mega variety

A TUI-based utility for real-time monitoring of InfiniBand traffic and performance metrics on the local node

llm-d is a Kubernetes-native high-performance distributed LLM inference framework

A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.

A lightweight design for computation-communication overlap.

Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport