Big Data Notes

Exploring big data technologies with Raspberry Pi.

View the Project on GitHub jdoldis/big-data-notes

Rasberry Pi Cluster

Why

To do distributed computing you need multiple computers. A cheap option for building a cluster is to use Raspberry Pis.

Equipment

Quantity Item Amazon Link
3-4 Raspberry Pi Raspberry Pi
1 per pi Micro SD Card SD Card
1 per pi USB A to USB C Cables Cable Pack
1 per pi Ethernet Cable Ethernet Cable
1 USB Wall Charger Wall Charger
1 Cluster Case Cluster Case
1 Micro SD Card Reader Reader

The ethernet cables aren’t required if you just want to use wifi, but wired connections are more reliable. If you do order them be sure to check that your router has sufficient ethernet ports. If not, look into getting an ethernet switch.

Photos

Cluster Pic 1 Cluster Pic 2 Cluster Pic 3

Setup

There’s already a ton of guides about how to set up Raspberry Pis so I’ll keep this very high level:

  1. Format the SD Card: This involves putting the operating system onto the SD card. Essentially you just download the raspbian OS and write it to the card. You’ll need a micro SD card reader to do that. See the docs for more details.
  2. Configure the pi: It’s easiest to do this using the GUI. This requires a keyboard, mouse and monitor (or TV) to be plugged into the pi. If you can’t do this look into a headless setup.