Object Tracking Using Object Tracking Using Autonomous Quad Autonomous Quad

CopterCopter

Carlos A. MuñozCarlos A. Muñoz

Robotics, Intelligent Sensing & Control (RISC) Laboratory,

School of Engineering, University of Bridgeport,

221 University Avenue, Bridgeport, CT 06604, USA.

What is it, exactly?What is it, exactly?• A way to catch (tap) an object in the air.• Coordinates gather from Kinects• Automatically “sees” the object • Computes end destination• Sends signals to move the quad copter

Hardware UsedHardware Used

3D Robotics Quad Copter

Two Microsoft Kinects

Overall FlowOverall Flow

How does it know How does it know what the object is?what the object is?

• Object recognition Emgu CV:• Image converted to black and white • HSV image values• Slides determine final intensity threshold value• HSV values are added to the program• Red box shows detection of the object(s)

HSV upper and lower HSV upper and lower limit selectorslimit selectors

Adding object Adding object Object recognition EmguCV:Image converted to black and white HSV image valuesSlides determine final intensity threshold valueObject is added to programRed shows detection

How does it track it?How does it track it?Front cameraFront camera

Blue rectangle quad copter area

Green is the minimum horizontal distanceGreen plus red is the maximum horizontal distance (1/8 )

Red lines are the vertical distances at different time intervals

How does it compute How does it compute the XY destination the XY destination

values? values? • Spot the object at a time interval [t]• Calculate initial horizontal velocity • Calculate initial vertical velocity

Vx = x/t Vy = (y/t) – 0.5*g*t 2

Calculate different horizontal displacements until it reaches the copter.

X = Vx * t and Y = Vy*t+ 0.5*g*t 2

• Use that same t to find the vertical displacement.• Determine new XY based on those values

What about the Z What about the Z value?value?

• Bottom Kinect tracks two time intervals (uses same times )• Calculates the final location using the distance formula 

dx = t2x – t1x dy = t2y – t1y

 Do the square of the distance between dx and dy: 

(dx2 + dy2) * k2 = tfinal (h) Solve for k then get the final point: 

x3 = x1 + t1x + dx*k y3 = x1 + t1y + dy*k

How does it control How does it control the copter? the copter?

• Camera tracks copter like an object• Virtual joystick created (Vjoy)• keyboard to joystick mapper configured• Program simulates keyboard presses• Once the new XYZ is acquired, execute the key

presses

.ini file

Mission planner joystick configuration

Does it really work?Does it really work?

• Limitations:• Air resistance • Quad must be stabilized (minimal movement)• Quad is sensitive and very fast (crashed it 3

times)• Similar objects with color that is close to object

Questions?Questions?• Websitehttps://sites.google.com/site/carlosmrobotics/home

• Email carlosm@my.bridgeport.edu