SGLang is a fast serving framework for large language models and vision language models.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

FlashMLA: Efficient Multi-head Latent Attention Kernels

Make SGLang go brrr

Production-ready platform for agentic workflow development.

OpenAI ChatGPT, GPT-3, GPT-4, DALL·E, Whisper API wrapper for Go

DeepEP: an efficient expert-parallel communication library

LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.

OpenAI 接口管理 & 分发系统，支持 Azure、Anthropic Claude、Google PaLM 2 & Gemini、智谱 ChatGLM、百度文心一言、讯飞星火认知、阿里通义千问、360 智脑以及腾讯混元，可用于二次分发管理 key，仅单可执行文件，已打包好 Docker 镜像，一键部署，开箱即用. OpenAI key management & redistributi…

Unify Efficient Fine-Tuning of 100+ LLMs

A community Swift package used to interact with the OpenAI API

BERT classification model for processing texts longer than 512 tokens. Text is first divided into smaller chunks and after feeding them to BERT, intermediate results are pooled. The implementation …

An opensource agent orchestration framework built on top of the latest OpenAI Assistants API.

🤩 宝~点两下，我就能骗走你所有的 Star！

Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

This repository contains a Python implementation that allows you to use gorilla-llm/gorilla-openfunctions-v2 LLM to perform function calling using the OpenAI protocol. It provides a way to extend t…

github release、archive以及项目文件的加速项目

收集免费的接口服务,做一个api的搬运工

Manage github releases from the command line

Workflow engine for Kubernetes

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

OpenAIOS vGPU scheduler for Kubernetes is originated from the OpenAIOS project to virtualize GPU device memory.

A CLI interface for Marp and Marpit based converters