“pickabo” · 2014. 12. 9. · “pickabo” chaudhari anup s december 9, 2014 university of...
Post on 22-Aug-2020
0 Views
Preview:
TRANSCRIPT
“Pickabo” Chaudhari Anup S
December 9, 2014
University of Florida
EEL 5666 – Intelligent Machines Design Lab (IMDL)
Instructors: Dr. A. Antonio Arroyo, Dr. Eric M. Schwartz
Teaching Assistants: Andy Gray, Nick Cox
Table of Contents Abstract
…………………………………………………………………………………………………………1
Executive Summary
…………………………………………………………………………………………………………1
Introduction
…………………………………………………………………………………………………………1
Integrated System
…………………………………………………………………………………………………………2
Mobile Platform
…………………………………………………………………………………………………………3
Actuation
…………………………………………………………………………………………………………3
Sensors
…………………………………………………………………………………………………………3
Detailed Description of the Special Sensor System
…………………………………………………………………………………………………………4
Behaviours
…………………………………………………………………………………………………………5
Experimental Layout and Results
…………………………………………………………………………………………………………6
Conclusion
…………………………………………………………………………………………………………6
Appendix
…………………………………………………………………………………………………………7
Abstract: The project aims towards building an autonomous robot that will be able to search for red
coloured balls, grab them and take them towards the base stations.
Executive Summary: To achieve the expected objective the robot must be able to perform following tasks:
1. Navigate in a room in search of ball.
2. Recognize the ball and its colour.
3. Grab the ball using a mechanical system.
4. Search for the base station and go towards it to drop the ball.
Introduction: Pickabo is an autonomous robot intended to detect and grab red coloured spherical objects.
Pickabo is primarily controlled by an Arduino Mega but all of the vision processing is
handled by an on-board HP ENVY J001TX laptop running a version of Ubuntu. Locomotion is
provided by two independent motors and everything is powered by a 5000 mAh lithium
polymer battery. Pickabo uses a webcam to detect the red spheres and ultrasonic sensors to
avoid obstacles. Pickabo identifies the targets using algorithms native to OpenCV.
This report details the hardware used on Pickabo and provides a high-level description of
Sentinel’s software and behavior architecture. Descriptions of the tests performed as well as
ideas for future work are provided at the end. Finally, full copies of the software can be
found in the appendices.
Integrated System:
Integrated System
Pickabo has two main processors on board.
1. The main processor is an HP ENVY J001TX laptop which is responsible for vision
processing and behavioural algorithms.
HP ENVY J001TX
2. The other processor is an Arduino Mega 2560. It has a serial communication with the
laptop and is used to send commands to the drive motors and the grabber servo.
The Arduino also reads inputs from the ultrasonic sensors and relay that information
to the laptop.
Arduino Mega 2560
Mobile Platform: The platform of the robot consists of four wheels. Two of these wheels are driver wheels
(hooked to DC motors) that allow it to use differential drive to move in its surroundings. These
wheels are situated towards the front of the platform. The other two wheels are caster wheels
situated towards the back of the platform. This allows for simple control of the robot while
still maintaining stability.
Actuators: 1. 2 DC motors for differential drive.
Arduino motor shield which lets you drive two DC motors with Arduino board,
controlling the speed and direction of each one independently.
2. Servo motor for grabbing mechanism.
Sensors: 1. Pickabo uses a web camera to capture a real time video.
PlayStation 3 Eye
PlayStation 3 Eye
2. Pickabo has three Ultrasonic sensors to determine the distance of obstacles from the
bot.
PING))) Ultrasonic Sensor
PING))) Ultrasonic Sensor
Detailed Description of the Special Sensor System: Machine vision is one of the most effective and low-cost methods for detecting any object.
Pickabo’s special sensor is the PlayStation 3 Eye web camera that allows it to perform the
task of locating coloured balls. While the other sensors allow it to drive around and avoid
obstacles, it is the webcam that allows Pickabo to collect critical data about its environment.
The webcam, ultrasonic sensors and actuators work in unison in order for Pickabo to make
intelligent decisions and complete the objective.
The vision processing is handled by the laptop running Ubuntu 14.04 LTS and an algorithm
developed using OpenCV 2.4.10. The algorithm first captures a temporary image i.e. a frame
from the video and converts it from BGR to HSV. Then it eliminates all other colours from the
frame except red by image thresholding. Hough circle transform is used for detecting circles
in the frame. It also determines the radius as well as the coordinates of the centre of circle.
The laptop then sends commands to the Arduino via serial communication according the
coordinates of the centre.
Image Processing
Behaviours: 1. Search the ball:
At first, the robot starts revolving about itself in search of a ball at the location where
it started. The web cam will check if it sees any ball irrespective of its colour, every
time it pauses revolving for a short time. The bot starts to align with the ball after
detecting it.
2. Approach towards ball and grab it: Once the robot has found a ball, it needs to approach towards it. It does this by first
centring the ball in the view. Once it centres the ball, it will drives straight towards it.
Every now and then, it checks again to make sure the ball is still at the centre of the
view and adjust if it isn’t. Also, it constantly checks the distance of the ball in terms of
its radius in the frame. Once it reaches a distance where the ball is close enough, it
grabs the ball by the front claw.
3. Search for the base station: Once the bot grabs the ball, it will starts revolving at that place in search of a base
station. Once the base station is found, the bot starts approaching towards it.
4. Approach towards base station and drop ball: Once the base station has been found, the bot keeps approaching towards it by
keeping it at the centre of the view. During the approach, it constantly checks the
distance of the base station in terms of distance between the centroids of two
different strips on it. Once the distance increases to a certain specified value the bot
stops and drops the ball.
Experimental Layout and Results: Originally, Pickabo was going to use an Odroid-U3 as a processor. But due to my lack of
experience in this field I ended up using my laptop as a processor. The chassis of the robot
had to be redesigned to accommodate the bigger and bulkier processor on-board.
Conclusion: Pickabo was my first full-fledged robotics project. So building it was quite an enlightening
experience. There is however a large amount of room for improvement, particularly in the
area of enabling Pickabo to detect and differentiate between spheres according to their
colours as originally intended. The obstacle avoidance behaviour couldn’t be integrated with
the main code. So, it is something that can be worked upon.
Appendix:
Image processing code #include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc_c.h"
#include <stdlib.h>
#include <stdio.h>
#include <iostream>
#include <string.h>
#include <unistd.h>
#include <stdint.h>
#include <fcntl.h>
#include <termios.h>
#include <errno.h>
#include <sys/ioctl.h>
#pragma comment (lib, "opencv_core2410d.lib")
#pragma comment (lib, "opencv_highgui2410d.lib")
#pragma comment (lib, "opencv_imgproc2410d.lib")
#pragma comment (lib, "opencv_video2410d.lib")
#pragma comment (lib, "opencv_features2d2410d.lib")
using namespace cv;
using namespace std;
int main(int argc, char** argv)
{
char mode;
char buffer[2]; //Character buffer for data string sent from the ODROID to the Arduino
int fd; //The device label for the Arduino
struct termios toptions; //Store the serial port settings
//Open the serial port
fd = open("/dev/ttyACM0", O_RDWR | O_NOCTTY);
printf("fd opened as %i\n", fd);
usleep(3500000);
// Wait for the Arduino to reboot
tcgetattr(fd, &toptions);
//Get current serial port settings
//Set 9600 baud both ways
cfsetispeed(&toptions, B9600);
cfsetospeed(&toptions, B9600);
// Serial settings: 8 bits, no parity, no stop bits
toptions.c_cflag &= ~PARENB;
toptions.c_cflag &= ~CSTOPB;
toptions.c_cflag &= ~CSIZE;
toptions.c_cflag |= CS8;
// Canonical mode
toptions.c_lflag |= ICANON;
//Commit the serial port settings
tcsetattr(fd, TCSANOW, &toptions);
VideoCapture cap(1); //capture the video from webcam
if (!cap.isOpened()) // if not success, exit program
{
cout << "Cannot open the web cam" << endl;
return -1;
}
namedWindow("Control-1", CV_WINDOW_AUTOSIZE); //create a window called
"Control"
namedWindow("Control-2", CV_WINDOW_AUTOSIZE); //create a window called
"Control"
int iLowH1 = 0;
int iHighH1 = 15;
int iLowH2 = 160;
int iHighH2 = 179;
int iLowS1 = 149;
int iHighS1 = 255;
int iLowS2 = 149;
int iHighS2 = 255;
int iLowV1 = 59;
int iHighV1 = 255;
int iLowV2 = 59;
int iHighV2 = 255;
//Create trackbars in "Control" window
createTrackbar("LowH1", "Control-1", &iLowH1, 179); //Hue (0 - 179)
createTrackbar("HighH1", "Control-1", &iHighH1, 179);
createTrackbar("LowH2", "Control-2", &iLowH2, 179); //Hue (0 - 179)
createTrackbar("HighH2", "Control-2", &iHighH2, 179);
createTrackbar("LowS1", "Control-1", &iLowS1, 255); //Saturation (0 - 255)
createTrackbar("HighS1", "Control-1", &iHighS1, 255);
createTrackbar("LowS2", "Control-2", &iLowS2, 255); //Saturation (0 - 255)
createTrackbar("HighS2", "Control-2", &iHighS2, 255);
createTrackbar("LowV1", "Control-1", &iLowV1, 255);//Value (0 - 255)
createTrackbar("HighV1", "Control-1", &iHighV1, 255);
createTrackbar("LowV2", "Control-2", &iLowV2, 255);//Value (0 - 255)
createTrackbar("HighV2", "Control-2", &iHighV2, 255);
//Capture a temporary image from the camera
Mat imgTmp;
cap.read(imgTmp);
//Create a black image with the size as the camera output
Mat imgLines = Mat::zeros(imgTmp.size(), CV_8UC3);;
while (true)
{
Mat imgOriginal;
bool bSuccess = cap.read(imgOriginal); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
Mat imgHSV;
Mat imgThresholded;
Mat imgThresholded1;
Mat imgThresholded2;
cvtColor(imgOriginal, imgHSV, COLOR_BGR2HSV); //Convert the captured
frame from BGR to HSV
inRange(imgHSV, Scalar(iLowH1, iLowS1, iLowV1), Scalar(iHighH1, iHighS1,
iHighV1), imgThresholded1);
imshow("Thresholded Image 1", imgThresholded1);
inRange(imgHSV, Scalar(iLowH2, iLowS2, iLowV2), Scalar(iHighH2, iHighS2,
iHighV2), imgThresholded2);
imshow("Thresholded Image 2", imgThresholded2);
bitwise_or(imgThresholded1, imgThresholded2, imgThresholded); //Overlap
two thresholded images
//morphological opening (removes small objects from the foreground)
erode(imgThresholded, imgThresholded,
getStructuringElement(MORPH_ELLIPSE, Size(5, 5)));
dilate(imgThresholded, imgThresholded,
getStructuringElement(MORPH_ELLIPSE, Size(5, 5)));
//morphological closing (removes small holes from the foreground)
dilate(imgThresholded, imgThresholded,
getStructuringElement(MORPH_ELLIPSE, Size(5, 5)));
erode(imgThresholded, imgThresholded,
getStructuringElement(MORPH_ELLIPSE, Size(5, 5)));
double threshold = 100;
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
vector<int> small_blobs;
double contour_area;
Mat temp_image;
// find all contours in the binary image
imgThresholded.copyTo(temp_image);
findContours(temp_image, contours, hierarchy, CV_RETR_CCOMP,
CV_CHAIN_APPROX_SIMPLE);
// Find indices of contours whose area is less than `threshold`
if (!contours.empty()) {
for (size_t i = 0; i<contours.size(); ++i) {
contour_area = contourArea(contours[i]);
if (contour_area < threshold)
small_blobs.push_back(i);
}
}
// fill-in all small contours with zeros
for (size_t i = 0; i < small_blobs.size(); ++i) {
drawContours(imgThresholded, contours, small_blobs[i], cv::Scalar(0),
CV_FILLED, 8);
}
//hough transform works well with blurred image
GaussianBlur(imgThresholded, imgThresholded, Size(9, 9), 2, 2);
vector<Vec3f> circles;
HoughCircles(imgThresholded, circles, CV_HOUGH_GRADIENT, 1,
imgThresholded.rows / 4, 100, 40, 10, 70);
for (size_t i = 0; i < circles.size(); i++)
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
bool ball = false;
//Calculate the moments of the thresholded image
Moments oMoments = moments(imgThresholded);
double dM01 = oMoments.m01;
double dM10 = oMoments.m10;
double dArea = oMoments.m00;
//calculate the centroid of the white region
int posX = dM10 / dArea;
int posY = dM01 / dArea;
Point p1(posX, posY);
//cout << "\ncentroid :"<<Mat(p1)<< endl; //Print the centroid
//compare the centroid found by calculating moments with the center
found by hough transform
if ((fabs(posX - circles[i][0])<10) && (fabs(posY - circles[i][1])<10)) ball
= true;
else ball = false;
if (ball)
{
// circle center
circle(imgOriginal, center, 3, Scalar(0, 255, 0), -1, 8, 0);
// circle outline
circle(imgOriginal, center, radius, Scalar(0, 0, 255), 3, 8, 0);
cout << "\nI found a ball..." << "\ncenter : " << center <<
"\nradius : " << radius << endl;
if (radius <= 64)
{
if ((circles[i][0] - 320) > 15)
{
mode = 'r';
cout << "turning right to align with ball" <<
endl;
}
else if ((circles[i][0] - 320) < -15)
{
mode = 'l';
cout << "turning left to align with ball" << endl;
}
else
{
mode = 'f';
cout << "approaching towards the ball" << endl;
}
}
else if (radius >64 && radius<=70)
{
mode = 'g';
cout << "engaging the gripper to grab the ball" << endl;
}
}
else
{
mode = 'n';
cout << "\nI'll keep looking..." << endl;
}
}
imshow("Thresholded Image", imgThresholded); //show the thresholded
image
namedWindow("Hough Circle Transform Demo", CV_WINDOW_AUTOSIZE);
imshow("Hough Circle Transform Demo", imgOriginal);
if (waitKey(30) == 27) //wait for 'esc' key press for 30ms. If 'esc' key is
pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
sprintf(buffer, "%c", mode); //Construct data string
cout << buffer << endl; //Display the data string for debug
purposes
write(fd, buffer, 2); //Sent the data string to the Arduino
usleep(100000); //Slow data transmission rate to
accommodate the Arduino
}
return 0;
}
Arduino Code #include <Servo.h> //include the servo library to control the RobotGeek Servos
#define MICRO_SERVOPIN 10 //pin that the micro servo will be attached to
Servo microServo; //create an servo object for the 9g FT-FS90MG micro servo
const int speed = 255;
const int
PWM_A = 3,
DIR_A = 12,
BRAKE_A = 9,
SNS_A = A0;
//Motor A is the RIght Side motor
const int
PWM_B = 11,
DIR_B = 13,
BRAKE_B = 8,
SNS_B = B0;
//Motor B is the Left Side motor
void setup()
{
// Configure the A output
pinMode(BRAKE_A, OUTPUT); // Brake pin on channel A
pinMode(DIR_A, OUTPUT); // Direction pin on channel A
// Configure the B output
pinMode(BRAKE_B, OUTPUT); // Brake pin on channel B
pinMode(DIR_B, OUTPUT); // Direction pin on channel B
microServo.attach(MICRO_SERVOPIN);
microServo.write(150); // sets the servo position to 150 degress, positioning the servo for the
gripper open
Serial.begin(9600);
}
char mode;
void loop()
{
mode=Serial.read();
if(mode=='l') left(); // THE BOT MOVES IN LEFT DIRECTION
else if(mode=='r') right(); // THE BOT MOVES IN RIGHT DIRECTION
else if(mode=='f') front();// THE BOT MOVES IN FORWARD DIRECTION
else if(mode=='g') gripper(); // THE BOT ENGAGES THE GRIPPER
else if(mode=='n') look(); // THE BOT LOOKING FOR HOUGH CIRCLE
Serial.flush();
}
void left()
{
Serial.println("turning left to align with ball");
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_A, LOW); // setting direction to LOW the motor will spin forward
analogWrite(PWM_A, speed); // Set the speed of the motor, 255 is the maximum value
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_B, LOW); // setting direction to HIGH the motor will spin backward
analogWrite(PWM_B, speed); // Set the speed of the motor, 255 is the maximum value
delay(20);
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(100);
}
void right()
{
Serial.println("turning right to align with ball");
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_A, HIGH); // setting direction to LOW the motor will spin backward
analogWrite(PWM_A, speed); // Set the speed of the motor, 255 is the maximum value
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_B, HIGH); // setting direction to HIGH the motor will spin forward
analogWrite(PWM_B, speed); // Set the speed of the motor, 255 is the maximum value
delay(20);
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(100);
}
void front()
{
Serial.println("approaching towards the ball");
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_A, LOW); // setting direction to HIGH the motor will spin forward
analogWrite(PWM_A, speed); // Set the speed of the motor, 255 is the maximum value
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_B, HIGH); // setting direction to LOW the motor will spin forward
analogWrite(PWM_B, speed); // Set the speed of the motor, 255 is the maximum value
delay(80);
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(100);
}
void gripper()
{
microServo.write(70); //set gripper to 150 degrees = fully closed
//delay(200); //wait 3 seconds
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(50000);
}
void look()
{int dir = random(0,2);
if (dir ==0)
{
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_A, LOW); // setting direction to LOW the motor will spin forward
analogWrite(PWM_A, speed); // Set the speed of the motor, 255 is the maximum value
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_B, LOW); // setting direction to HIGH the motor will spin backward
analogWrite(PWM_B, speed); // Set the speed of the motor, 255 is the maximum value
delay(20);
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(100);
}
if (dir == 1)
{
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(BRAKE_A, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_A, HIGH); // setting direction to LOW the motor will spin backward
analogWrite(PWM_A, speed); // Set the speed of the motor, 255 is the maximum value
digitalWrite(BRAKE_B, LOW); // setting brake LOW disable motor brake
digitalWrite(DIR_B, HIGH); // setting direction to HIGH the motor will spin forward
analogWrite(PWM_B, speed); // Set the speed of the motor, 255 is the maximum value
delay(20);
digitalWrite(BRAKE_A, HIGH); // setting brake LOW disable motor brake
digitalWrite(BRAKE_B, HIGH); // setting brake LOW disable motor brake
delay(100);
}
}
top related