Software Design Process

Introduction

Along with the mechanical team, we researched the movements necessary to create distinct latte art patterns. After outlining the roles our 3-axis gantry and tilting stepper motor would play when creating basic designs, we came to understand that it was necessary to establish complete control over the machine’s movements as multiple of them move at a time, as well as be precise when planning the motor paths. We also agreed that designing a clean graphic user interface would not only make the user experience better when interacting with our robot, but aligned with our goals for creating a professional project.

We started with the following large learning goals:

  • Control the motors and plan paths wirelessly with a Raspberry P
  • Create a GUI with a website in React
  • Communicate with the Raspberry Pi wirelessly using Flask
  • Create a GUI feature where a user can design their own latte art pattern
  • Convert an image into Latte art

Research G-Code

We began our process by researching G-Code, since most 3D printers and CNC machines use G-Code to communicate and control the machine. We looked into the Grbl library and tried implementing it, but ultimately decided to create our own G-code library. This decision was based on a multitude of reasons, such as alignment with learning goals, feedback given by Olin peers, and getting to have better understanding and full control of our robot. Creating a G-Code library aligned with our learning goals to practice developing better code architecture, especially with Arduino. In addition, we reached out to an Olin student who had created a G-code library for their project. He helped give us a broad understanding of G-Code and how to implement it. Our final and most impactful reason for choosing to create our own G-Code library was to fully customize the robot’s movements. Latte art requires a lot of complex and unique movements that we wanted to fully control. By creating our own G-Code library, we would gain a strong understanding of the underlying firmware that controls the motors and the overall structure to communicate paths from software to firmware.

Create Basic G-Code Library

We started researching how to create our own G-Code library and looked through past PIE project implementations of G-Code. Ultimately, we found an article explaining how to build a 2 axis CNC interpreter using G-Code.

While the electrical team was learning how to establish connections with stepper motors, we worked on understanding the basics of the G-code library we found. We rewrote the code to control 4 LEDs as a start.

Through creating a G-Code library, we learned how to change and make new G-Code commands, how to read and parse G-code, and establish serial connection between Python and Aurdrino as well as Python functions to send G-code. In Python, we used functions to send G-code over serial. While we were creating the G-code library, we were also learning how to control stepper motors.

Controlling Stepper Motors

Once we decided to create our own G-Code library, we decided to switch our focus to learning how to control the stepper motor movements. The mechanical subteam took apart a 3D printer until we had a 2-axis gantry with two stepper motors to start testing out code. We initially started testing with a single stepper motor, and researched different stepper motor drivers in Arduino or associated with the electrical components like the shield or drivers.

When we were able to consistently move the motors as expected in one step, one revolution or continuously, we calibrated the system to learn how many degrees it takes to move 1 mm. We decided to use the metric of 1 mm to measure each step and command because we could easily measure and create designs in terms of mm, and it was a small enough step to be able to move in patterns without losing precision. During this process, we worked closely with the electrical team learning how to control and wire stepper motors.

After calibrating and learning how to control the motor, we were able to draw a line by coding the motors to move a certain distance of mm. Once we accomplished moving one motor in a certain distance, we quickly added the second motor to draw squares.

At this point, we also transformed the LED G-code library such that the functions no longer controlled the LED blinks, but the motors themselves. New functions were added as well for other motor functionality The first iteration of a square was in relative coordinate systems where the commands told the motors where to move. Once we completed a square in relative motion, we added global coordinates where the firmware updated the motors current located based on the commands, and found the distance to move by subtracting the new location by the old location. To test our code beyond watching the motors move, we added a sharpie and cardboard to draw squares as seen in the photo below.

Our goal for controlling the 2D gantry was to be able to draw a circle, since latte art typically requires a base foundation of milk that is circled into the espresso before a design is started.

Unlike drawing a square, a circle required drawing curves which proved more difficult than we initially anticipated. We researched different algorithms including bresenham's line algorithm which created a curve by moving each motor individually in alternating small steps based on a conditional statement, but ultimately chose to use MultiDriver to move multiple motors at once.

StepperDriver, the Arduino library we used to control the motors, has a function called MultiDriver which moves up to three motors simultaneously. We chose to use MultiDriver as opposed to bresenham's line algorithm because we wanted to move the machine in the most efficient manner. Zoomed in, Bresenham's line algorithm would create a stair like pattern, while MultiDriver would create a hockey puck shape. If one motor had to move significantly further than the other, both motors would drive at the same time and speed until one hits its designated spot and will stop while the other motor keeps moving

Once the firmware was ready to draw curves, we created a function called make_circle that would automatically generate coordinates for a circle with a certain radius size and return an imputed number of coordinates. Once the coordinates were generated we were able to draw a circle.

Create G-Code for the Latte Art Machine

With a good understanding of G-Code commands, we started to add more components including adding the limit switches to the code, and creating a go_home function that moved all the motors to a known position where the limit switches were activated. Once the mechanical gantry was completed, we added additional stepper motors for the Z axis, tilt mechanism and additional y-axis stepper motor. We also added in G-Code to control the solenoid valve.

When starting to transition to Python-to-Arduino Serial communication for controlling motors, we had to ensure that G-Code understanding was established on both sides; Python must be able to generate the proper G-Code commands as we tell it to, while Arduino must be able to parse it and control the motors as desired.

As we began planning paths, we added more G-Code for more features or changes. One important shift was enabling and disabling motors between runs. We noticed the Y axis and tilt motors were overheating despite not actively moving in the current path, so we started disabling motors as a default and only enabling them before usage. The original concern with this was that disabling certain motors such as the Z axis motor would cause whatever is attached to it to fall and hit the cups. The mechanical team made it possible to avoid this by adding a lead screw to hold the Z axis up even while the motor was disabled, and a tilt mechanism that would swing to a stable position when power was cut.

A major change we made to the firmware was being able to move all 5 stepper motors simultaneously. We edited multiple files in the StepperDriver library which initially only moved 3 stepper motors at the same time. Towards the end of the project, we cleaned up the naming system to match G-Code convention, and added comments and docstrings to the code, so everyone on the team could help test the path planning.

Path Planning

We started researching how to convert a png to coordinates to G-Code, but we soon realized that we could not write an entire latte path based on an image. We were able to automate the generation of G-Code commands for 2D shapes like a circle and sine wave, but ultimately had to code the majority of the G-Code paths by writing in exact coordinates of where we wanted the robot to go to, and continually making fine adjustments and tweaks when running the code.

We brainstormed multiple paths and what motions would be required to mimic the latte art of creating a base, heart, and rosetta. We set these as minimal viable products with the hope of moving onto more complex shapes like a swan, tulip, and horse. While planning the paths, we created more G-Code commands to fill actions we did not already include like speed. Once the paths were drafted, we tested them on the full system.

Testing with Full Mechanical Setup

While testing, we found multiple bugs in the G-Code and planning as well as mechanical and electrical systems. We started testing the paths with dry runs to make sure the motion of the machine was what we expected, then moved onto a water run to prevent bad spills and lastly to steamed milk with a cup with espresso. We fine-tuned the path as we tested, and after every test, we talked about what could be improved to help create the shape. We created the base pattern and heart pattern initially before working on the rosetta as we fine tuned the heart pattern. We also redid the architecture of the Python code for all the functions and communication to go in one file, each path to be an individual file, and a file to choose which path to run.

GUI

We initially wanted to use Raspberry Pi for the GUI and communicate between the Arduino and Raspberry Pi with a website that uses Flask to wirelessly communicate the paths. A learning goal was to learn how to use a Raspberry Pi, create a web application and use Flask but unfortunately due to time constraints and the steep learning curve of using a Raspberry Pi, coding in React and using Flask, we decided to use a Pygame, a Python library we already had experience with.

We wanted to create an aesthetic GUI as well as a logo, so we brainstormed color scheme and general formatting ideas on Canva, a free design tool, as well as creating multiple flow charts for the GUI. We designed the GUI elements on Canva as well, then moved the buttons, images, and backgrounds we generated to Pygame. We pulled on knowledge of class architecture from previous classes to design functionality classes for the GUI buttons and page switching to ultimately create our final GUI.

Next Steps

Although the GUI worked on certain laptops, we began experiencing file path errors during our final demonstration. In the future, we would like to address this issue by having a cleaner file system. Moreover while brainstorming, we created stretch goals for the GUI like adding animation while the user was waiting for the latte to arrive or adding fun latte facts. We plan on including those features in future iterations. In addition, with more time we would love to shift our focus to using a Raspberry Pi and React website with Flask. A Raspberry Pi would prevent the need of a laptop being so close to a liquid and create a cleaner overall look since anyone could order from their phone. We would also continue fine-tuning the paths and include more designs. Lastly, we would add a developer feature which would allow the user to control certain G-Code movements through arrow and keyboard clicks. This would help with testing along with users the ability to create their own latte art.