[ICLR 2024] Lemur: Open Foundation Models for Language Agents

🐙 OctoPack: Instruction Tuning Code Large Language Models

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

A multi-programming language benchmark for LLMs

Code and documentation to train Stanford's Alpaca models, and generate the data.

Code for the paper "Evaluating Large Language Models Trained on Code"

CLI platform to experiment with codegen. Precursor to: https://lovable.dev

A playbook for systematically maximizing the performance of deep learning models.

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

An implementation of a deep learning recommendation model (DLRM)

Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

Code repo for "Language Models with Transformers" paper

Transformer implementation in PyTorch.

Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training

[ICML 2020] Obtaining Adjustable Regularization for Free via Iterate Averaging

Fair Resource Allocation in Federated Learning (ICLR '20)

Decentralized SGD and Consensus with Communication Compression: https://arxiv.org/abs/1907.09356

Synchronized Batch Normalization implementation in PyTorch.

Simple Hierarchical Count Sketch in Python

PyTorch for benchmarking communication-efficient distributed SGD optimization algorithms

A PyTorch implementation of the paper "Training Neural Networks Using Features Replay"

Code for Federated Learning with Matched Averaging, ICLR 2020.