Communication

Our software system utilized ROS to communicate between the camera, laptop, and Arduino. The ROS ecosystem allowed us to have many separate scripts on different pieces of hardware that can communicate with each other. These individual nodes use ROS to publish and subscribe to data. ROS also helped us create visualizations that could be viewed on multiple computers on the same network.

Camera System

We analyzed images from our camera using the OpenCV library, which allowed us to track where the puck was on the table and also track the edges of the table using fiducial markers.

The puck was located using HSV filtering. Originally, we had issues with the puck reflecting lights from the ceiling; at different points in the table, it would appear as a different color, because the lighting was inconsistent. This was fixed by adding a matte blue vinyl layer to the pucks that did not reflect the light.

Six fiducial markers were applied to known locations around the table. The apriltag library was used to locate the tags and compute their IDs. A minimum of four markers is necessary to compute the puck location in the table space; a total of six were included because the tags are prone to being incidentally covered by the player or the puck.

The puck is originally detected in the coordinate system of the camera (camera space). The fiducials are used to create a homograph matrix which allows us to transpose the puck location into the table coordinate system (table space). The homography matrix is computed by OpenCV, which makes the transformation simple.

The final transformation becomes

Trajectory Predictions

By keeping track of the puck position over time, we were able to calculate the x and y velocity vectors of the puck and use those to extrapolate where the puck's future position would be. In order to account for bounces off the edges of the table, we assumed that all collisions would be perfectly inelastic (i.e. the puck would bounce at the exact same angle it hit the edge of the table at). As we were only initially working in one axis, we calculated the point in which the puck would cross the line y = 12, since the single axis gantry laid on the table 12 inches away from the x-axis. If this point was off the table (i.e. the puck was aimed toward a wall), we then reflected the point of intersection back onto the table recursively to place the predicted position in the correct location.

We implemented our model of table physics by projecting the velocity in a straight line.

To account for the puck bouncing off the walls, we kept the projection in a straight line and instead reflected the puck position across the table walls if it was out of bounds.

We also applied low pass filters to the puck's position to reduce the amount of jitter that the predicted position experienced.

Striker Control

We used the AccelStepper Arduino library to control our stepper motors that moved the striker. We set up our system to be a state machine with three different states: run, stop, and home. Once turned on, we started the game in stop mode. This mode disabled our motors to help with temperature control and waited to receive a position command from ROS. Once an initial position command was received, the state machine entered home mode. This involved enabling the motors and sending the striker to the left until it triggered the break beam sensor. The current location of the striker was set to be 0 and the state machine then entered run mode. Run mode was set up to receive position commands from ROS and send the striker to that location as quickly as possible. If the break in receiving position commands was ever longer than two minutes, the system entered stop mode.

Future Work

The next step for this project would be to implement control code to move the striker in two axes. This goal was unable to be reached during the timeframe of the project due to mechanical limitations. Two axes of motion would elevate the project as the puck would be able to be hit back to the human player (as opposed to just being blocked), and allows us to add another level of complexity to the project: gameplay. There are game simulations and machine learning algorithms that can be written to design the position commands in such a way that the robot is able to play offensively and score against a human opponent.

As we integrated our systems, we discovered issues where the motors skipped steps. When this happened, we needed to rehome the striker more frequently to ensure that its current position was accurate. In a future revision of this project, we would like to track the striker position using the camera and compare it to the position that the striker reported being at. If there was ever a large discrepancy, we could then command the striker to rehome itself and resolve the difference. This would allow the robot to be more autonomous and require less human input.