This document describes how to add code to RCCL.

RCCL uses Google C++ coding standards with few changes. One of them is, RCCL uses 4 space width indent. To make life easier, RCCL uses clang-format to automatically format C/C++ headers and source files. The .clang-format file is present in the root directory of the repo. Use clang-format on changed source files to format them before pushing them to repo. clang-format is shipped with clang packages, if rocm is installed in the machine, it is present in hcc binary directory /opt/rocm/hcc/bin. To automatically format a source file, do clang-format -i ./src/rccl.cpp (make sure you are in root directory of rccl, as it contains .clang-format configuration file). clang-format should be applied to .cpp, .h files in ./tests, ./inc and ./src directory.

RCCL code is modularized depending on the functionality. Each rccl op api is defined in a file, for example, rcclAllReduce api should be implemented in rcclAllReduce.cpp file. The implementation then call functions in rcclAllReduceRuntime.h, which contains logic to determine different launch parameters to launch kernels to finish allreduce operation. The kernels are defined in rcclAllReduceKernels.h, where all the copy and op are implemented to transfer and operate data from different gpus.