Demo

An Arduino Uno manages a user input panel that supports selecting, playing, and generating songs. A laptop with Google Magenta’s MultiTrackVAE machine learning model generates longer pieces based selected song excerpts and can send the contents of MIDI files over a serial connection. An Arduino Mega extracts note pitch, instrument, and duration information in real time. The visualized output manifests as dancing cats on various motors and linkage systems with a sparkling LED matrix as the backdrop.

Watch Demo

Team Goal

We prioritized both aesthetics and functionality - we wanted to combine our skills and interests to create a sound-based, visually interesting system that sparks joy!

Project Objectives

  1. Lovable: Cheerful and visually appealing design
    Cuteness = Top Priority
  2. Interesting: Utilize user input, generative ML, and real-time MIDI bitstream decoding to control mechanical and electrical outputs
  3. Easy-to-Use: Incorporate joyous music making for people at any music level

Integration Diagram

From user input to visual & audio output

Mechanical Components

Our mechanical team primarily used lasercutting and 3D printing to create the structure that expressed our output. There are 3 components to the build:

  1. The Cats: various linkage systems and stepper motor interactions make 3D-printed cats puppets dance. Four unique motions were incorporated into our design: (1) up and down, (2) the infinity loop, (3) dancing pair, and (4) one dancing on its own.
  2. The Stage: the primary housing unit for this project had to be large enough to hold our wires, 9 motors and stands to support the cats, the linkages that drive the cat motions, and a LED matrix. The housing unit is 5.5” in height and has an 18” diameter. The walls are created from living hinges and the components of the stage are held together with a slot-based design.
  3. The LED Housing: this component of the project brings height and acts as the primary expression of note pitches. It stands at 16.5” tall and holds 64 LEDs in an 8x8 matrix. The material is laser cut acrylic, assembled with a puzzle piece method and spray painted black to minimize the reflection of light in any unwanted directions.
Learn more

Electrical System

The LED Matrix + MAX7219 LED Driver Chip:
64 LEDs are hooked up in a parallel grid layout. The Arduino Mega interfaces with a MAX7219 LED driver chip to individually control each LED using a 3-pin SPI protocol. The MAX7219 is powered by the 5V power supply from the Arduino.

DC Stepper Motors + ULN2003 Driver Boards:
Nine 28-BYJ48 stepper motors with ULN2003 driver boards drive the dancing cat movements. Each stepper motor requires 4 I/O pins, using 36 pins in total. The Arduino Mega is a large Arduino board with enough I/O pins to control both the 9 motor drivers and the LED Matrix. We use a 12V 5A AC→DC power supply to power the ULN2003 driver boards' with their maximum supported voltage and provide all 9 stepper motors with ~260mA of current.

User Input Panel:
An Arduino UNO is hooked up to an input panel with 9 push buttons with 9 status LEDs.

Learn more

Software Subsystems

Our code system uses an Arduino Uno, an Arduino Mega, and a laptop with a Linux OS. The Arduinos were programmed in C++, and the laptop runs a Python3 script and bash commands. There are 4 subsystems:

User Input Manager

An Arduino Uno manages a button panel interface that allows the user to experience and select from eight short 2 second song excerpts. The user can also click a button to generate a new 60 second song based on the selected excerpts. The chosen action is read by the laptop via pySerial.

Learn more

LED Matrix and Motor Control

An Arduino Mega decodes the MIDI bitstream in order to extract pitch, instrument, and duration information. These properties are used to control the behavior of an 8x8 LED Matrix column and 9 stepper motors.

Learn more