Colossal-AI: A Unified Deep Learning System for Big Model Era

Ongoing research training transformer models at scale

Tensors and Dynamic neural networks in Python with strong GPU acceleration

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Build and train PyTorch models and connect them to the ML lifecycle using Lightning App templates, without handling DIY infrastructure, cost management, scaling, and other headaches.

Examples of training models with hybrid parallelism using ColossalAI

A collection of models built with ColossalAI

PKU一键出入校备案小工具 Github Action