A high-throughput and memory-efficient inference and serving engine for LLMs

A Datacenter Scale Distributed Inference Serving Framework

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

AWS MCP Servers — helping you get the most out of AWS, wherever you use MCP.

NVIDIA Inference Xfer Library (NIXL)

Client implementation of the MQTT 3.1.1 specification for embedded devices