SGLang is a high-performance serving framework for large language models and multimodal models.

🍻 Default formulae for the missing package manager for macOS (or Linux)

Containerization is a Swift package for running Linux containers on macOS.

Docker Model Runner

Run LLMs with MLX

A dynamic library providing Virtualization-based process isolation capabilities

Source repo for Docker's Documentation

MLX: An array framework for Apple silicon

Ubuntu Bootc experiment with composefs-native backend

Linux kernel source tree

Community maintained hardware plugin for vLLM on Apple Silicon

Tensors and Dynamic neural networks in Python with strong GPU acceleration

A high-throughput and memory-efficient inference and serving engine for LLMs

LLM inference in C/C++

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

A small self-contained alternative to readline and libedit