Our code base is split into two parts - vision software and Arduino firmware.
The vision code runs on a Raspberry Pi running Ubuntu 20.04 and identifies a target in
frame. The centroid of the target is then identified and its offset from the center of
the image is sent over serial to the Arduino. Then the Arduino control loop takes the
offset as input and uses it to adjust the robot's speed. If no input is given, the robot
should continue in the same direction that it was taking at the time of the last input.
There are two separate methods in which the strandbeest operates -
tracking a color target and tracking a person. In either case, the
code will identify the target of choice in the frame, and calculate the
offset of the target from the center of the frame. This value is sent
over serial to interact with the Arduino code.
One difference in operation between color tracking and person tracking -
if a person is not found in frame, the code will send a halt signal to the
Arduino. Alternatively if the color target is lost, no signal is sent to the
Arduino, causing the beest to continue in the same direction as it was prior.
Arduino Firmware Overview
The Arduino software powers the beest at a constant forward speed unless given
input over serial. Upon being given an offset value, the right and left motor
speeds are updated by adding and subtracting respectively the offset multiplied
by a scalar value. The motors are powered at these speeds until new input is
given.
If an offset of exactly 0 is received, all motors will be stopped,
including the rear one. Once a non-zero offset is received, the motors are repowered
at the base running speed with the offset modifier.
Video Evidence
The video shows person tracking in action, as you can see it doesn't always work perfectly.
The algorithm sometimes picks up the light in the staircase and reflection as a person rather
than the person in frame. However it is able to handle another person in frame who is closer and
switches to track them instead.
