Hand of God

About

The main task for the software in our project was to predict the position on the gantry where the ball was going to land. We used webcams and openCV to track the position of the ball. We used curve fitting to predict where the ball would land.

Tracking the Ball

In order to track the position of the ball, we used openCV that would read from the two webcams that we had. Our ping pong ball was bright green and we used color masking to track the position of the ball. We were able to get the coordinates of the centroid of the ball whenever it was in frame of either of the cameras through this method

Demonstration of Software Tracking Ball

Camera Layout

To track the ball, we are using two webcams. One camera is on the size and tracks the horizontal motion. The other camera sits on the table facing upward and tracks motion in the lateral direction. In the image above, one can see the side camera on the whiteboard.

Tracking in the Y Direction

We used our side view camera to track where the ball was going in the y-direction. After the camera got the first three points along the motion of the ball, we would fit a parabola onto the points to visualize where it would land. As the ball continues to move forward, the parabola continues to update based on the new position of the ball. We found the y-coordinate of the hand in the camera frame. Once we had a parabola, we would find the x-coordinate of the parabola at the place where it intersects with the y-coordinate of the hand to know how far the ball will travel in the y direction and predict the y-coordinate. Based on some testing, we decided that we should predict the y position after the camera has found 12 centroids as that gives us an accurate prediction. If we waited much longer, then the hand won't move fast enough to catch the ball.

Tracking in the X Direction

We used our top view camera in order to calculate the x position of the ball. The camera was placed below looking up and when the ball was thrown over it, the coordinates of the ball on the camera show a straight line that indicates what angle the ball is being thrown at. We used this angle and the distance of the camera from the gantry to predict the x position.

Once we had both the x and y coordinates, we could send those points over to the motors in order to move the hand along the gantry.

Sending Position to Gantry

# top_x = trackingFunctions.find_x(theta, predicted_y, cam_dist) # top x is x and side x is y from drawing  
predicted_x = x_start + x_change  
print("finished top prediction")             
print("Predicted x: ", predicted_x, "y: ", predicted_y)
MoveMotors(arduino, (int(predicted_x), int(predicted_y)))
return True