An interactive DJ pad that uses hand gesture detection to trigger and control beats

https://editor.p5js.org/Christinawu/sketches/MRVp6HKH9

Summary

I created an interactive beats pad using the HandPose model in p5.js, allowing users to control and trigger sounds through hand gestures. The system detects whether the hand is open or closed and responds when an open hand hovers over one of four virtual rectangles displayed on the screen. Each rectangle corresponds to a different sound: bass, drum, guitar, and synth. When the hand is open and positioned over a specific rectangle, the associated sound plays, and the rectangle's transparency changes to indicate activation. The layout and interaction design make it easy to control the DJ pad by sitting directly in front of the screen.

Inspiration

I was inspired by an app I used to play in middle school called “SUPER PADS DJ”. It’s a phone app that lets you play famous beats and create your own music. Inspired by it, I wanted to create something similar in p5.js using the HandPose model.

Process

Before I start to write codes, I first found some free beats online. I found those beats on

https://freesound.org/. After finding these beats, I moved on to work on the codes.

I first started working with the HandPose keypoints example code because it provided the hand's keypoints. The first challenge I needed to address was how to place rectangles in the center of the screen and spread them out evenly. Initially, I attempted to work with six rectangles, but I found it difficult due to the screen's limitations, requiring me to sit far away for my hand to fit inside each square. As a result, I decided to use four rectangles instead, making it easier to control while sitting in front of the computer. I positioned the four rectangles evenly and set their default transparency to 127, or about 50%.

function drawSquares() {
  let squareSize = 200;
  let margin = 20;
  let x = width / 2 - squareSize - margin / 2;
  let y = height / 2 - squareSize - margin / 2;

  noStroke();

  fill(255, 105, 180, squareTransparency[0]);
  rect(x, y, squareSize, squareSize, 20);

  fill(62, 180, 137, squareTransparency[1]);
  rect(x + squareSize + margin, y, squareSize, squareSize, 20);

  fill(135, 206, 250, squareTransparency[2]);
  rect(x, y + squareSize + margin, squareSize, squareSize, 20);

  fill(255, 255, 0, squareTransparency[3]);
  rect(x + squareSize + margin, y + squareSize + margin, squareSize, squareSize, 20);
}