Source: This repository is based on the overview of the official Docker Apache Hadoop image.

Prerequisites: Docker and Docker Compose installed on your machine.

Clone or download this repository. Then, build and run the containers using Docker Compose:

docker compose up -d --build

Log into a node via:

docker exec -it namenode bash

Access the namenode UI via: http://localhost:9870

Access the ResourceManager UI via: http://localhost:8088

Run an example MapReduce job to test the setup: First log into the namenode as specified above, then run:

yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.5.jar pi 10 15

Note: The version number in the path may vary based on the installed Hadoop version (3.5.5 in this case).

Shut down the cluster via:

docker compose down