TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet and so on. 文本生成模型，实现了包括LLaMA，ChatGLM，BLOOM，GPT2，Seq2Seq，BART，T5，UDA等模型的训练和预测，开箱即用。

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.