Achieve state of the art inference performance with modern accelerators on Kubernetes

LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

Kubernetes-native Job Queueing

Sample applications for Google Kubernetes Engine (GKE)

llm-d benchmark scripts and tooling

Production-Grade Container Scheduling and Management

llm-d helm charts and deployment examples

A high-throughput and memory-efficient inference and serving engine for LLMs

helm charts for deploying models with llm-d

GenAI inference performance benchmarking tool

An Extensible Deep Learning Library

Enhancements tracking repo for Kubernetes

Kubernetes website and documentation repo:

This repository contains the website implementation and content for the GKE AI Labs site. GKE AI Labs is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and …

SGLang is a fast serving framework for large language models and vision language models.

JobSet: a k8s native API for distributed ML training and HPC workloads

Code and configuration to manage Kubernetes project infrastructure, including various *.k8s.io sites

Test infrastructure for the Kubernetes project.

Meta configuration for Kubernetes Github Org

Autoscaling components for Kubernetes

Large Language Model Text Generation Inference

A simple, performant and scalable Jax LLM!

Add Google Cloud Platform support to Terraform