opencv on raspberry pi _ let's make robots!

OpenCV on Raspberry PiLogin or register to post comments by Ladvien Collected by 7 usersTip/walkthrough

By Ladvien@ April 24, 2013Raspberry Pi, Camera, Arduino, and Servo tracks face

Attachment Sizefacedetectv10.zip4.29 KB

Start HereRulesTipsRobotsSomething elseBlogsReviewsChallengesForumsShopAboutDonate

Home

SearchSearch page

User login

Username: *

Password: *

Create new account

Request new password

Log in using OpenID

Recent blog postsSneak peek at theArduino Robot

Pig 'rastlin (Or, How to beannoyed...)

We could be the thirdsending a humanoidrobot to Space

Club Jameco Entry

Win a Cheapduino Set

Guangzhou ScienceCenter robot competition

Weekend Project: EitherCreate a 3-Axis Printer orFinish My Delta Robot

Drive Stepper (4th axis)off of PWM SpindleControl Ouput

Stepping up/MovingForward...

theory vs. soldering

more

Recently submittedlinks:

Build your own FPGA

CoasterBot!!! RobotDesign Contest by ClubJameco

os for AVR ATMEGASand ATTINY.

US Congress Roadmapfor Robotics report

Communica (Pty) Ltd

Walker robot inspiration.

The Big Brain Theory$5000 Challenge

New electronics supplywebsite

Not the Doctor Watsonyou'd usually imagine

OpenCV on Raspberry Pi | Let's Make Robots! http://letsmakerobots.com/node/36947

1 of 9 6/20/2013 5:37 PM

No longer afeared of frying my Pi, I've moved on to trying to implement some of my bot goals. Like many, I want my bot to be able tointeract with people, but I didn't realize that I'd stumble on this ability.

I've looked at many visual processing boards like the CMUcam v4, but I'm not paying $100 for any board. I looked into making one, itlooks possible, but not much cheaper. So, I got curious as to what alternatives there are. I stumbled on Hack-a-Day's recommendedarticle: OpenCV on Raspberry Pi.

Anyway, he provided instructions on setting up OpenCV (open source computer vision) on Raspberry Pi. Of course, it was about 20minutes later I had the code working on my Pi.

I had been skeptical of the Pi's ability to run any computer vision software, and morever, it's usefulness given the Pi's processingconstraints. But once I had it up and running, I noticed it actually ran smoother than I had hoped. Don't get me wrong, I think it is lessthan 10FPS, but I could tell it would work for many robot applications More than that, if the Raspberry Pi was used only for thecomputer vision, then it would still be cheaper than many other hardware driven CV boards.

Basic Raspberry Pi and WiFi Dongle

WiFi Dongle: $6.17Raspberry Pi: $35.00SD Card (4g): $2.50Web cam: $8.00

Total for Basic RPi: $51.67

Therefore, I went to work on hacking his code.

Many hours later, I ended up with a very crude Raspberry Pi, Ardy, Camera, and Servo orchestration to track my face. Mind you, this isa proof of concept, nothing more at this point. But I hope to eventually have my bot wandering around looking for faces.

Image of Pi VNC. The box outline is being written through i2c.

Pulling apart a little $8 eBay camera.

To Setup the Raspberry Pi:

If you're setting it up from sratch, start with these instructions.

But if you're already setup, I think all you need is OpenCV.

(Cancer ResearchMachine)

The ultimate robotcomputer

Who's onlineThere are currently 21 usersand 11 guests online.

Online usersbananic

bdk6

Dan M

Duane Degn

Fred van Donk

birdmun

Martini

kariloy

Chris the Carpenter

Malerun

Ladvien

bdk6

bananic

Maxhirez

jscottb

Space Goat

mogul

Gearmadness

Bajdi

studikasus

mogul

Quick linksLMR on Google+

LMR on Facebook

LMR on Flicker

LMR on Twitter

LMR Scrapbook

Pages

RSS feeds


2 of 9 6/20/2013 5:37 PM

$ sudo apt-get install python-opencv

The Code:

The Arduino code reads bytes from the i2c, converts them to characters, then places the characters into an integer array. The Pi issending 4 numbers, 2 coordinates, x1, y1, x2, y2.

The Python code is "facetracker.py" by Roman Stanchak and James Bowman, I've merely added lines 101-105, which load thecoordinates of the box around your face into a a string, converts that to a string array. I also added function txrx_i2c(). This functionconverts the string array into bytes and sends it to the i2c bus.

To change this setup from i2c to UART, focus on the txrx_i2c() in the Python code and the onRead() in the Arduino code. I assure you,UART would be much easier.

If anyone has any questions hollar at me. Oh! And if someone can tell me ways I could optimize this code, I'm all ears.

The Arduino Code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

#include <Wire.h>#define SLAVE_ADDRESS 0x2A#include <Servo.h>

Servo CamServoX; //Attach the pan servo.Servo CamServoY; //Attach the tilt servo.

int ServoTimer = 250; // Change to adjust how quickly the servos respond.

int SmallXJump = 3; //Sets the movement amount for small pan jumpsint LargeXJump = 7; //Sets the movement amount for large pan jumps

int SmallYJump = 1; //Sets the movement amount for small pan jumpsint LargeYJump = 2; //Sets the movement amount for large pan jumps

//How close your face is to the edge to trigger a jump.int SmallYLimit = 40; int LargeYLimit = 20;

int SmallXLimit = 40;int LargeXLimit = 20;

//Set servos to initial position.int posX = 90; //Servo position.int posY = 90; //Servo position.

int x1; int y1;int x2; int y2; //Holders for frame dimesions.

// Indexes for getting i2c bytes, then, converting them to integers.int i = 0;int varI = 0;

//Sets flag to trigger ServoWrite() from the main loop.//I tried to put this under 'onRequest' call, but the Raspberry Pi kept giving me errors.//This flagging was a work around.int NoServoData = 0;

int dim[12]; //Char array for char[] ---> int conversion.char d[8]; // Char holder array for byte-->char conversion.

void setup() { // initialize i2c as slave Wire.begin(SLAVE_ADDRESS); Wire.onRequest(sendData); Wire.onReceive(readData); Serial.begin(9600); //Attach servos CamServoX.attach(10); //Tilt (Y) CamServoY.attach(9); //Pan (X) //Write initial servo position. CamServoX.write(posX); CamServoY.write(posY);}


3 of 9 6/20/2013 5:37 PM

57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136

void loop() {

//Again, this is the work around. The flag "NoServoData" is set under the i2c onReceive.if (NoServoData==1){ ServoWrite();}

}

//This is just to show the RPi can be written to. //Replace with stuff you want to write to the Pi.char data[] = "Pasta"; int index = 0;

// callback for sending datavoid sendData() { Wire.write(data[index]); ++index; if (index >= 5) { index = 0; } } // callback for receiving data.void readData(int numbytes) {

//Holds the charsint c;

if (Wire.available() > 0){ while(Wire.available()) // slave may send less than requested c = Wire.read();} //Add each integer to a char array. //Skip commas ',' and keep adding the integers until char '\0' is received. //Then print out the complete string. if (c != ','){ if(c != '\0'){ d[i] = d[i] + c; //Appends the characters to an array. i++; } } else{ i=0; //Reset the d char array index. if(varI < 7){ //We only want to get integers until we get all four numbers (x1, y1, x2, y2) plus dim[varI]=atoi(d); //Convert the d int into ASCII and store it in the dim array. d[0]=0;d[1]=0;d[2]=0;d[3]=0;d[4]=0;d[5]=0; //Clear the d array (i2c doesn't like for loops in this function varI++; //Increase dim index. } else{ //We now have all four numbers, load them into the variables. x1=int(dim[4]); y1=int(dim[1]); x2=int(dim[2]); y2=int(dim[3]);

NoServoData = 1; //Set the WriteServo() call flag. varI=0; //Reset the dim index to prepare for next set of numbers. } i=0; //Reset some }}

void ServoWrite(){ int x3 = 160 - x2; // Calculate the distance from the right edge of the screen int y3 = 120 - y2; // Calcualte the distance

//For X Axis if(x1 < SmallXLimit ){ //Only do small jumps, since not too far away from the edge. if(posX>1){ //If the pan servo is at its edge, do nothing. for (int i = 0; i < LargeXJump; i++){ posX++; // Set the new position CamServoX.write(posX); //Make the adjustment. delay(ServoTimer); //Delay between servo increments. } } }


4 of 9 6/20/2013 5:37 PM

137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216

if(x3 < SmallXLimit){ if(posX<180){ for (int i = 0; i < LargeXJump; i++){ posX--; CamServoX.write(posX); Serial.println(posX); delay(ServoTimer); } } }

if(x1 < LargeXLimit){ if(posX>1){ for (int i = 0; i < SmallXJump; i++){ posX++; CamServoX.write(posX); Serial.println(posX); delay(ServoTimer); } } } if(x3 < LargeXLimit){ if(posX<180){ for (int i = 0; i < SmallXJump; i++){ posX--; CamServoX.write(posX); Serial.println(posX); delay(ServoTimer); } } }

//For Y Axis if(y1 < SmallYLimit ){ if(posY>1){ for (int i = 0; i < SmallYJump; i++){ posY--; CamServoY.write(posY); Serial.println(posY); delay(ServoTimer); } } } if(y3 < SmallYLimit){ if(posY<180){ for (int i = 0; i < SmallYJump; i++){ posY++; CamServoY.write(posY); Serial.println(posY); delay(ServoTimer); } } }

if(y1 < LargeYLimit){ if(posY>1){ for (int i = 0; i < LargeYJump; i++){ posY--; Serial.println(posY); CamServoY.write(posY); delay(ServoTimer); } } } if(y3 < LargeYLimit){ if(posY<180){ for (int i = 0; i < LargeYJump; i++){ posY++; CamServoY.write(posY); Serial.println(posY); delay(ServoTimer); } }


5 of 9 6/20/2013 5:37 PM

217218219220221

}

//Reset servo write flag.NoServoData=0;}

Now for the Python Code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

#!/usr/bin/python"""Have to execute using "sudo python facedetect.py --cascade=face.xml 0"(Normal build sudo python "%f")This program is demonstration for face and object detection using haar-like features.The program finds faces in a camera image or video stream and displays a red box around them.

Original C implementation by: ?Python implementation by: Roman Stanchak, James Bowman"""import sysimport cv2.cv as cvfrom optparse import OptionParserimport timeimport threadingimport readlineimport pygamefrom pygame.locals import *import sysimport smbus

# Parameters for haar detection# From the API:# The default parameters (scale_factor=2, min_neighbors=3, flags=0) are tuned # for accurate yet slow object detection. For a faster operation on real video # images the settings are: # scale_factor=1.2, min_neighbors=2, flags=CV_HAAR_DO_CANNY_PRUNING, # min_size=<minimum possible face size

min_size = (20, 20)image_scale = 2haar_scale = 1.2min_neighbors = 2haar_flags = 0

"""i2c Code"""bus = smbus.SMBus(1) # Open up a i@C bus.address = 0x2a # Setup Arduino address

sendstring = "" # This will be my send variable (RPI-to-Arduino)bytearraytowrite = [] #Actual array for holding bytes after conversion from string.

#This function actually does the writing to the I2C bus.def toWrite(a):

global sendstringglobal bytearraytowritebytearraytowrite = map(ord, sendstring) #This rewrites the string as bytes.for i in a:

bus.write_byte(address, i)

def txrx_i2c():global sendstring#while True:sdata = "" rdata = ""for i in range(0, 5):

rdata += chr(bus.read_byte(address));#print rdata#print bytearraytowrite#print "".join(map(chr, bytearraytowrite)) #Will convert bytearray to string.

#Writes the key commands to the i2c bus.toWrite(bytearraytowrite)

#time.sleep(.6);

def detect_and_draw(img, cascade): global sendstring


6 of 9 6/20/2013 5:37 PM

71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150

# allocate temporary images gray = cv.CreateImage((img.width,img.height), 8, 1) small_img = cv.CreateImage((cv.Round(img.width / image_scale),

cv.Round (img.height / image_scale)), 8, 1)

# convert color input image to grayscale cv.CvtColor(img, gray, cv.CV_BGR2GRAY)

# scale input image for faster processing cv.Resize(gray, small_img, cv.CV_INTER_LINEAR)

cv.EqualizeHist(small_img, small_img)

if(cascade): t = cv.GetTickCount() faces = cv.HaarDetectObjects(small_img, cascade, cv.CreateMemStorage(0), haar_scale, min_neighbors, haar_flags, min_size) t = cv.GetTickCount() - t print "detection time = %gms" % (t/(cv.GetTickFrequency()*1000.)) if faces: for ((x, y, w, h), n) in faces: # the input to cv.HaarDetectObjects was resized, so scale the # bounding box of each face and convert it to two CvPoints pt1 = (int(x * image_scale), int(y * image_scale)) pt2 = (int((x + w) * image_scale), int((y + h) * image_scale)) cv.Rectangle(img, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0) x1 = int(x * image_scale) y1 = int(y * image_scale) x2 = int((x + w) * image_scale) y2 = int((y + h) * image_scale) sendstring = str(x1) + "," + str(y1) + "," + str(x2) + "," + str(y2) + "," sendstring = sendstring.translate(None, '() ') print sendstring txrx_i2c() sendstring = "" cv.ShowImage("result", img)

if __name__ == '__main__':

parser = OptionParser(usage = "usage: %prog [options] [filename|camera_index]") parser.add_option("-c", "--cascade", action="store", dest="cascade", type="str", help="Haar cascade file, default %de (options, args) = parser.parse_args()

cascade = cv.Load(options.cascade) if len(args) != 1: parser.print_help() sys.exit(1)

input_name = args[0] if input_name.isdigit(): #Where the image is actually captured from camera. "capture" is the variable holding image. capture = cv.CreateCameraCapture(int(input_name)) else: capture = None

cv.NamedWindow("result", 1)

width = 160 #leave None for auto-detection height = 120 #leave None for auto-detection

if width is None: width = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH)) #Gets the width of the image. else: cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_WIDTH,width) #Gets the width of the image.

if height is None:height = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT))

else:cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_HEIGHT,height)

if capture: #If "capture" actually got an image. frame_copy = None while True:

frame = cv.QueryFrame(capture) if not frame: cv.WaitKey(0) break if not frame_copy:


7 of 9 6/20/2013 5:37 PM

151152153154155156157158159160161162163164165166167168169170171

frame_copy = cv.CreateImage((frame.width,frame.height), cv.IPL_DEPTH_8U, frame.nChannels)

# frame_copy = cv.CreateImage((frame.width,frame.height),# cv.IPL_DEPTH_8U, frame.nChannels)

if frame.origin == cv.IPL_ORIGIN_TL: cv.Copy(frame, frame_copy) else: cv.Flip(frame, frame_copy, 0) detect_and_draw(frame_copy, cascade)

if cv.WaitKey(10) >= 0: break else: image = cv.LoadImage(input_name, 1) detect_and_draw(image, cascade) cv.WaitKey(0)

cv.DestroyWindow("result")

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.By nhbill@ Sun, 2013-04-28 14:25

You are getting me hungryLogin or register to post comments

for some Raspberry PI! That is so cool!

All you need is a voice recognition module...

By Ladvien@ Sun, 2013-04-28 17:02

In the worksLogin or register to post comments

That's my next project. :)

Well, I should say it's on the "list."

By Bajdi@ Sun, 2013-04-28 12:58

I've thought about adding anLogin or register to post comments

I've thought about adding an rpi to my hexapod. Since I have all the hardware and my new chassis is big enough. But I would need toadd another big battery to power the rpi. At the moment I use a 3S 3000mAh Lipo to power the legs, it only lasts 10 minutes... I think I'llredesign my Rover 5 and add the rpi to that. I'm also very interested in the new Beagle Black. Looks like a better platform for robotics.

By Ladvien@ Sun, 2013-04-28 15:25

Ah.Login or register to post comments

Ah. I know so very little (in general and) about hexapods.

I believe a Beaglebone Black will be my next purchase. Although, the single USB port concerns me greatly. I've seen manyinconsistencies with USB hubs.

But more than the Black, I'm waiting for Parellella (at least the 16 core). As I understand it, it would allow real time CV, with mutlipleobject recognition, on a hobby budget ($99).

By Maxhirez@ Wed, 2013-04-24 20:25


8 of 9 6/20/2013 5:37 PM

LeelaLogin or register to post comments

Since you have a little more anthropism (did I make that word up? I intentionally didn't use anthropomorphism) to your Leela than I didmine, can I make a special request? I'd love to see at least one picture of it with a purple yarn ponytail stuck on it! Is the camera onUSB or another bus? I hope I can find some time to play with your code. In the meantime, there's a new Beaglebone for $45. I'llprobably pick one up when I get some steady work.

By Ladvien@ Thu, 2013-04-25 04:34

Found thisLogin or register to post comments

http://roboteurs.com/beaglebone-black-vs-raspberry-pi/

I'm starting to think I might try a Beagle Black. Seems like for $10 extra you get an DAC and an extra 300mhz.

By Ladvien@ Wed, 2013-04-24 21:14

Of course, sir.Login or register to post comments

I hear you on the work. Found out yesterday our grant won't be renewed in Sept.

Oh well.

After my wife braids my pony-tail tonight I'll ask her to braid up some purple yarn.

Hadn't seen that Beagle yet. I still think TI's presence in the openhardware community seems forced. Like an old farty CEO tryingto keep up with customers on Facebook. It really comes out in their promotion gir's voice towards the end, "What will YOU? create."

Still, I won't say a damned thing more against them. Lord knows, I've got a stack of samples on my desk with their name all overthem, which they graciously mailed to me, even paying post.

I thought about doing this with my Pi and overclocking to 1ghz, instead of opting for a different board (e.g., pcDuino).

I hope you're well, sir.

By Bajdi@ Wed, 2013-04-24 19:52

Damn it's time I finish myLogin or register to post comments

Damn it's time I finish my hexapod and start playing with my rpi. This is so cool :)

By Ladvien@ Wed, 2013-04-24 21:17

Hexapods are beyond me...Login or register to post comments

But, how 'bout a Bajdi Hexapi? :)

ALL LMR ARE BELONG TO US!Let's make robots!


9 of 9 6/20/2013 5:37 PM

opencv on raspberry pi _ let's make robots!

Documents