Portland State University Portland State University

PDXScholar PDXScholar

Undergraduate Research & Mentoring Program Maseeh College of Engineering & Computer Science

5-2018

Real-time Object Detection And Tracking On Drones Real-time Object Detection And Tracking On Drones

Tu Le Portland State University

Follow this and additional works at: https://pdxscholar.library.pdx.edu/mcecs_mentoring

Part of the Materials Science and Engineering Commons, and the Robotics Commons

Let us know how access to this document benefits you.

Citation Details Citation Details Le, Tu, "Real-time Object Detection And Tracking On Drones" (2018). Undergraduate Research & Mentoring Program. 25. https://pdxscholar.library.pdx.edu/mcecs_mentoring/25

This Poster is brought to you for free and open access. It has been accepted for inclusion in Undergraduate Research & Mentoring Program by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: pdxscholar@pdx.edu.

Real-time object detection and tracking on dronesTu Le, Ehsan AryafarPortland State University

Motivation

Next generations of commercial and military drones are expected to be aware of surrounding objects while flying autonomously in different terrains and conditions. One of the biggest challenges to drone automation is the ability to detect and track objects of interest in real-time. While there are many robust machine learning algorithms for object detection and tracking, these algorithms may not perform as expected on drones due to low computing power system and weight constraints.

Objective

Implement machine learning algorithms for the drone to perform real-time object detection and tracking using on-board camera and low-power embedded system.

Equipments

Methodology

The authors acknowledge the support of the Semiconductor Research Corporation (SRC) Education Alliance (award # 2009-UR-2032G) and of the Maseeh College of Engineering and Computer Science (MCECS) through the Undergraduate Research and Mentoring Program (URMP).

Conclusion and Future work

Detection algorithms are accurate but slow and computationally expensive. Tracking algorithms are often very fast but not accurate, especially if fast moving objects and jump cuts are present. Therefore, a smooth handshake between the two is necessary to improve overall performance.Future work includes evaluations and implementations on NVIDIA Jetson TX2 which is a newer version of embedded system for drones.

Acknowledgements

References

1. “DJI - The Future Of Possible.” DJI Official, https://www.dji.com. Accessed 23 May 2018.

2. Young, Eric, and Frank Jargstorff. Image Processing & Video Algorithms with CUDA. 2008, p. 60.

3. Parekh, Himani S., et al. A Survey on Object Detection and Tracking Methods. Vol. 2, no. 2, 2007, p. 9.

Contact

Tu Leletu@pdx.edu

DJI Matrice 100 DJI Manifold

DJI Matrice 100 with DJI Zenmuse X3 camera.DJI Manifold built on top of NVIDIA Tegra K1 which is a 32-bit architecture and low power onboard computer.

Data transmission:The flight controller, Manifold, and camera are connected together using 8-pin and 10-pin cables.Raw video stream data extracted from the camera are converted from YUV into RGB pixel values using the following formula:

Detection and tracking system design:Detector runs once a few frames and gives object detection result to update the tracker which runs continuously every frame to track the known object. The delay between detector’s runs can be adjusted based on the task or the accuracy threshold set for the tracker.

…

…