torchax is a PyTorch frontend for JAX. It gives JAX the ability to author JAX programs using familiar PyTorch syntax. It also provides JAX-Pytorch interoperability, meaning, one can mix JAX & Pytor…

Muon fsdp 2

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

Flash-Muon: An Efficient Implementation of Muon Optimzer

📄Small Batch Size Training for Language Models

Tools for understanding how transformer predictions are built layer-by-layer

An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

NanoGPT (124M) in 3 minutes

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

My solutions to leetGPU: https://leetgpu.com/challenges AND BEYOND

code for the ddp tutorial

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Welcome to the 'In Context Learning Theory' Reading Group

A professionally curated list of awesome Conformal Prediction videos, tutorials, books, papers, PhD and MSc theses, articles and open-source libraries.

This repository contains code for the paper "Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs", RecSys '24

An Open Source Implementation of Anthropic's Paper: "Towards Monosemanticity: Decomposing Language Models with Dictionary Learning"

Training Sparse Autoencoders on Language Models

A PyTorch native platform for training generative AI models

A LinearOperator implementation to wrap the numerical nuts and bolts of GPyTorch

18.337 - Parallel Computing and Scientific Machine Learning

Experiment ASsistance for Your Time-Series Forecasting, EasyTSF

C++ Crash Course

100 numpy exercises (with solutions)