Ongoing research training transformer models at scale

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Example models using DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

A library for efficient similarity search and clustering of dense vectors.

A fast, distributed, high performance gradient boosting (GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks…

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow

Mirror of Apache Hadoop

Mirror of Apache Spark

Spark Terasort

Java load client that utilizes the YCSB Framework and runs load tests against Memcached, Membase, and other database systems. I no longer work on this project, see distributed loadgen.

A simple, asynchronous, single-threaded memcached client written in java.

Yahoo! Cloud Serving Benchmark