Run multiple docker containers in parallel

Build docker image

# In main branch of sdk-platform-java
git pull
DOCKER_BUILDKIT=1 time docker build \
  --file .cloudbuild/library_generation/library_generation.Dockerfile \
  -t local:latest \
  --target final .

Prepare API definitions

# In master branch of googleapis
git checkout 48d30c4966ef9ea31b897e13f75d8f94070cc8e9

Run script

# In main branch of google-cloud-java
git checkout chore/parallel-gen
chmod +x parallel-run.sh
./parallel-run.sh

Explanation

The original generation configuration (generation_config.yaml) has been split to three files with roughly the same size.

The script will start three docker run command running in the background, each command will generate 1/3 libraries in one config.

Log of each execution will be sent to run-${num}.log in the current working directory.

Note that you may need to change the path to API definitions:

# Change the path to googleapis in your environment.
-v "$(pwd)/../googleapis:${workspace_name}/apis" \

time spent in parallel generation (cloudtop)

real 33m45.521s
user 0m0.258s
sys 0m0.499s

real 38m2.565s
user 0m0.248s
sys 0m0.580s

real 42m6.416s
user 0m0.308s
sys 0m0.480s

time spent in sequential generation (cloudtop)

real 67m28.459s
user 0m0.574s
sys 0m2.081s