Xorbits Inference(Xinference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models.

Python actor framework for heterogeneous computing.

xllamacpp - a Python wrapper of llama.cpp

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

A high-throughput and memory-efficient inference and serving engine for LLMs

Official enterprise version docs.

Pipeline Parallelism for PyTorch

This is a landscape of the infrastructure that powers the generative AI ecosystem

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant

Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, ChatGLM2)

Made pandas and SQL APIs work seamlessly together, with out-of-core and distributed capabilities.

Scalable Python data science, in an API compatible & lightning fast way.

Python SQL Parser and Transpiler

PandasAI is a Python library that integrates generative artificial intelligence capabilities into Pandas, making dataframes conversational

Python 分布式编程

✨ Build AI interfaces that spark joy

Chat language model that can interpret and execute functions/plugins

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Large Language Model Text Generation Inference

A curated list of awesome Python frameworks, libraries, software and resources

Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in dataset using pandas

Uses Auto-GPT with Llama.cpp