Benchmark suite for LLMs from Fireworks.ai

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Public repo for HF blog posts

Foundation model benchmarking tool. Run any model on Amazon SageMaker and benchmark for performance across instance type and serving stack options.

A high-throughput and memory-efficient inference and serving engine for LLMs

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

A universal scalable machine learning model deployment solution

Large Language Model Text Generation Inference

pkcs11 wrapper for Go

Development repository for the Triton language and compiler

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

JSON for Modern C++

CUDA Templates for Linear Algebra Subroutines

Lightweight C++ command line option parser

The NVIDIA® Tools Extension SDK (NVTX) is a C-based Application Programming Interface (API) for annotating events, code ranges, and resources in your applications.

Implement elliptic curve secp256k1

virtualpad package for ebiten

Extends `UICollectionViewFlowLayout` to support reordering of cells. Similar to long press and pan on books in iBook.

Extensible iOS and OS X animation library.

A repository of CocoaPods (cocoapods.org) specifications.

UIAppearance Stylesheets