Software

All code discussed in this section is available on our Github.

Image Processing

In order to represent an image using Skittles, each pixel has to correspond to a Skittle of a similar shade. We used yellow, orange, red, and purple Skittles, with the yellow Skittles representing the lightest pixels, orange Skittles representing medium-light Skittles, red Skittles representing medium-dark Skittles, and purple Skittles representing the darkest pixels.

First, the image is resized to fit the dimensions of the gantry bed (100 x 100 Skittles), then converted to grayscale. The image is then "paletted," meaning pixels with RBG values between 0 and 63 are assigned the RBG value 63, pixels between 64 and 127 are assigned the RGB value 127, and so on. This creates a new image composed of four different shades, each of which correspond to a different color of Skittle.

The original image (left) is downsized to the proper dimensions, grayscaled, and paletted, resulting in a simpler image (right).

Motion Protocol

We used Gcode commands to facilitate communication between firmware and software systems. Functions used to inform the motion protocol are detailed in the table below.
Function Arguments Description Calls Gcode Command(s)
check_for_ok ( ) None Waits for "ok\r" from firmware None None
home ( ) None Returns the end effector to (0, 0) check_for_ok G28
z_up ( ) None Moves end effector to "up" position check_for_ok G01 Z950
z_down ( ) None Moves end effector to "down" position check_for_ok G01 Z1400
refresh_feeder ( ) None Moves feeder in, then back out to refresh tray check_for_ok M411 / M410
go_to (x_dest, y_dest) - x: steps to take in x direction
- y: steps to take in y direction 		Moves end effector to given position check_for_ok G01 X[x] Y[y]
change_succ (v_state) - v_state: 0 to turn the vacuum off, 1 to turn it on Turns the vaccum on or off check_for_ok M400 / M401
retrieve (color) - color: Skittle to retrieve (yellow, orange, red, purple) Retrieves Skittle of a certain color go_to, z_down, z_up, change_succ None
complete_pixel (pixel_x, pixel_y) - pixel_x: steps to take in x direction
- pixel_y: steps to take in y direction 		Moves Skittle to its destination on the bed go_to, z_down, z_up, change_succ None
create_image (image_path) - image_path: path to image to make (str) Iterates through all pixels, retrieves the proper color Skittle, places it on the bed, then returns to (0, 0) home, retrieve, complete_pixel, refresh_feeder None

Next Steps

If given more time, I would have liked to implement a Skittle sorter using computer vision to automatically sort the Skittles by color so it would not need to be done by hand. I would have also liked to implement a Skittle recognition algorithm that would use computer vision to recognize Skittles and adjust the position of the end effector to pick them up. This would make for a much more impressive software component; however, given our limited budget, the hardware required to employ a mechanicsm like this would have been difficult to acquire.
Website Templates

created with

Website Builder Software .