Arduino and Raspberry Pi Controlled 3 DOF Robotic Arm for Color Based Sorting
As part of the research program at Bronx Science, I found a mentor at the NYU Tandon Mechatronics Lab to conduct my own project. This project was my first exposure to programming and electronics and my first interaction with Arduino and Raspberry Pi platforms.
The basis of this project was to create a program that could use color detection to sort a variety of objects. The Raspberry Pi would handle all color sensing and communicate to the Arduino to generate the motion of a 3 DOF robotic arm to properly sort the items.
Color Detection on the Raspberry Pi
OpenCV was the library used for computer vision here. To start off the process, color boundaries were defined based on the HSV Chart. This was a trial and error process where multiple attempts were needed to refine the exact range wanted for each color of interest. With the boundaries defined, a given image had to then be converted from the BGR color spectrum to HSV to match. The computer can then detect if any of the pixels in the image fit within a specific color range. If there are enough pixels of that color detected, we accept that the color is present in the image. The Raspberry Pi then converts the image into a mask for each color in black and white (white being where the color was detected, black being everything else). The masks serve as a visual for us to see what the computer is seeing.
For each item, the program cycles through a set of colors, checking to see if it is present in the image. If not, it moves on to the next color. This process resets for each item to be checked.
Initial tests were conducted on random images uploaded to the Pi. Once the tests were more successful, a Pi camera was utilized to take a picture of actual items and detect each of their colors so that they could be further sorted.
Arduino Code
Once the Raspberry Pi has detected a color, it sends its identification to the Arduino. The Arduino then checks what the result was and moves the servos on the robotic arm accordingly to place the item in the correct spot for its color.
To test that the Arduino was receiving the right inputs, an LED test was first performed before attaching the servos. The respective LED would light up whenever its corresponding color was detected. Note: Some LED colors did not directly correspond with the colors they were representing
Results
Some improvements are left to be made as the program has nuances to it when it comes to accurately detecting some of the colors. Environment, object type and specific color can each affect the ability for the Pi to detect a specific color.
