i

DREXEL UNIVERSITY

SUAS Aerial Surveillance Competition

Abstract Drexel’s system consists of an aircraft and a ground control station. The aircraft has a wingspan of 8.5 feet, and weighs approximately 20 lbs including payload. It is flown by the APM 2.5, an open source autopilot, which communicates to the ground via a 900 MHz 3DR radio. The aircraft system’s primary payload is a canon DSLR camera, mounted onto a two axis roll/pitch gimbal, which also actuates the camera’s zoom lens. The DSLR interfaces via USB to the onboard computer, the Pandaboard ES, which runs Ubuntu 11.10. The Pandaboard is responsible for taking pictures, tagging images with telemetry information, and uploading them to the ground control station over a 5.8 gHz Ubiquiti Bullet wifi module. It also performs onboard computer vision as the pictures are taken. This allows the aircraft to zoom in and take higher resolution photos while still in the targets’ vicinity. Drexel’s team is entering the full systems testing phase and has already performed successful demonstrations of prototype flights, camera interface and control, communications systems functionality, and computer vision algorithms.

Drexel’s team is advised by Dr. Ajmal Yousuff and funded by the Nasa Space Grant

ii

Contents 1 Engineering Methods ........................................................................................................................................................... 3

1.1 Requirements Analysis ............................................................................................................................................ 3

1.2 Design Decisions ..................................................................................................................................................... 3

1.3 Expected Performance .................................................................................................................................................. 3

2. System Overview ................................................................................................................................................................. 3

2.1 Team overview .............................................................................................................................................................. 3

2.2 Aircraft .......................................................................................................................................................................... 4

2.2.1 Airframe ................................................................................................................................................................. 4

2.2.2 Gimbal .................................................................................................................................................................... 8

2.3 Ground Control Station ................................................................................................................................................. 9

2.4 Communication Channels ............................................................................................................................................. 9

2.4.1 Data Channel ........................................................................................................................................................ 10

2.4.2 Autopilot Channel ................................................................................................................................................ 10

2.4.3 Safety Channel ..................................................................................................................................................... 10

2.5 Flight Hardware ........................................................................................................................................................... 10

2.6 Flight Software ............................................................................................................................................................ 10

2.7 Support Software ........................................................................................................................................................ 11

2.7.1 Mission Planner .................................................................................................................................................... 11

2.7.1 Gphoto2 ............................................................................................................................................................... 11

2.7.1 Gimbal Management and Aircraft Status ............................................................................................................ 12

2.7.2 Image Transfer and Sorting .................................................................................................................................. 13

2. 8 Automatic Detection/Cueing, Classification, and Identification System (ADCCI) ...................................................... 14

2.8.1 ADCCI Concept Development .............................................................................................................................. 14

2.8.2 ADCCI Embodiment/Detail Design ....................................................................................................................... 14

2.8.3 Image Stitching:.................................................................................................................................................... 17

2.9 Mission Planning ......................................................................................................................................................... 17

3. System Evaluation ............................................................................................................................................................. 17

3.1 Simulation ................................................................................................................................................................... 17

3.2 Bench Testing .............................................................................................................................................................. 18

3.3 Flight Testing ............................................................................................................................................................... 18

4. Safety ................................................................................................................................................................................ 19

5. References ........................................................................................................................................................................ 19

3

1 Engineering Methods

1.1 Requirements Analysis Drexel’s team chose to focus on key objectives and prioritize certain objectives above others to obtain a decent score as a first year competitor. Primary objectives revolved around reliable aircraft and communications systems, as well as an effective imagery platform. Secondary objectives included expanding the performance profile to fulfill autonomous target recognition and off-path pointing objectives. Tertiary objectives included autonomous takeoff/landing and SRIC capabilities.

1.2 Design Decisions In order to achieve a high mission score creating entirely new system from the ground up, emphasis was placed on payload functionality above aircraft efficiency and performance. The aircraft itself was treated more or less as a means to an end, able to carry the desired payload for the mission timeframe. A larger amount of time and money was spent on increasing payload capabilities.

1.3 Expected Performance Drexel’s team expects to meet primary and secondary objectives set forth, but neither of our tertiary objectives. A custom gimbal and DSLR camera will meet pointing and imagery requirements, while the onboard and ground computer systems have proven capable of autonomously classifying targets. Autonomous takeoff and landing will not be attempted, neither will SRIC.

2. System Overview

Figure 1. System Overview

2.1 Team overview Drexel’s Team consists of twelve senior year undergraduates comprising three separate senior design teams. Three are electrical engineering majors, and the remaining nine are mechanical engineering majors.

4

2.2 Aircraft Before we begin talking about the detail design methodology of our aircraft and gimbal, there are a few blanket methodologies that we used and are applied in their own way to each of the design aspects. The first stance we took is in recognizing the danger of developmental testing process of the autopilot. For this reason we have decided to place extra emphasis on the measure of reproducibility of each component. If a crash happens shortly before the competition deadline we want to be able to fix or replace most components within a day.

2.2.1 Airframe

2.2.1.1 Wing Design For our final plane, our wings were constructed from extruded polystyrene foam and included two long metal spars. The wingspan of our aircraft was 102” (8.5 ft) and included a Clark Y airfoil with a 17” chord length. Our wings were constructed using a simple wire cutting technique. It is important restate that it was not our team’s goal to design the most efficient aircraft. Three of our needs included maintaining stable flight, controlling flight, and controlling takeoff/landing, but as long as we met the technical specifications associated with those needs, we did not spend time continually optimizing the aerodynamics of our aircraft at the expense of tackling our more important stakeholder needs. As previously mentioned, our team essentially existed to serve the needs of the two other teams in our group, and fulfilling their needs dictated how our time was spent. There were many factors to consider when designing our wing. The following were the most important factors we are took into account in selecting the characteristic of wings. Our initial prototypes were constructed with foam board and had a hollow structure. Balsa wood was considered due to its prevalence in the modeling community. We constructed the wings for our final plane from extruded polystyrene foam. We made this decision for a number of reasons. It was the lightest material aside from balsa wood and was easily accessible. We machined it using a simple hot-wire cutting technique. Reproducibility was a key design feature for most components of our airframe. Due to our large wingspan, our wings were built in sections and this created obvious stress points. Stiffeners were necessary to make sure the wing did not collapse under the stress of flight.

Specs Weight Insulation Foam Core Carbon Fiber Balsa Wood

Reproducibility 20% 4 4 2 3

Durability 20% 2 2 5 2

Complexity 15% 4 4 1 3

Weight 30% 4 3 3 5

Cost 15% 5 4 2 4

TOTAL SCORE 3.75 3.3 2.75 3.55 Table 1. Decision matrix for wing materials

To determine proper area of the wing and coefficient of lift, we first considered wing loading. Wing loading is defined as the loaded weight of the aircraft over the chord length times the wing span. Cubic wing loading actually gives a better indication of performance [9] and is defined by the equation:

5

The larger the wings are relative to the aircraft, the lower the wing loading, and the higher the lift potential will be. Wing loading also affects the takeoff speed requirement. Wing loading varies directly with required takeoff speed and thus higher wing loadings required higher takeoff speeds. High wing loadings are not a problem for large high-speed aircrafts such as the Boeing 707 (it has a wing loading around 32.6 oz/ft3 [10]) because the high takeoff speeds generate enough lift for the aircraft to make it off the ground. Since we are dealing with low speed RC planes, minimizing the wing loading was very important. For our project we sought a design with a wing loading of less than 10 oz/ft3, which is in range of most aerobatic aircraft [11]. Smaller wing loadings lead to aircraft that are easier to handle. For our aircraft we used a wingspan of 102” (8.5 feet) and a chord length of 17 inches. With these inputs, the cubic wing loading is 8.43 oz/ft3, lower than our requirement. For our first test plane we selected and constructed the NACA 2412 airfoil out of foam board. The NACA2412 is an efficient airfoil and is frequently used with Cessna aircraft. For our final competition plane we used the Clark Y airfoil. The Clark Y is also an efficient airfoil used widely in RC Planes. This airfoil was highly suitable to our needs because it has high lift-to-drag ratio for lightweight aircrafts, and its lower surface is nearly horizontal, which made construction simple.

Figure 2. Clark Y profile

2.2.1.2 Tail Design For our plane we went with the conventional low horizontal stabilizer, vertical rudder tail configuration (an inverted T-shape). It was the simplest configuration and very convenient for our purposes. When designing and dimensioning out our tail we tried to keep the horizontal tail volume coefficient between 0.5 and 0.7. The horizontal tail coefficient is calculated using the following equation:

For the vertical tail, we used the “vertical tail volume coefficient” to guide in dimensioning, keeping it within the range of 0.02 and 0.05. This metric is calculated with the following:

6

In terms of our final tail design, we used a NACA0012 for the horizontal stabilizer with a chord length of 15” and a span of 33.5”. Our vertical stabilizer, also a NACA0012, was 14.7” tall and had a trapezoidal scheme with a top chord of 9.24” and a bottom chord of 14.25”. The horizontal stabilizer was 30% of the total wing area while the vertical stabilizer was about 12% of our wing area.

2.2.1.3 Propulsion Choosing the right propulsion system for the aircraft is very important. If a system is underpowered, the vehicle cannot fly, but if the plane is overpowered, it can become unstable. The takeoff weight of our aircraft is 22 lbs. This weight was determined using the SolidWorks mass properties tool and experimentation with our final aircraft. From this weight we can find a motor that will work using a power to weight ratio. In small aircraft design, there is a rule of thumb for the power to weight ratio: 50 W/lb for slow flying airplanes, 75 W/lb for sport airplanes, and 100 W/lb for acrobatic airplanes. We are designing our aircraft to fly somewhere between a slow flying and sport aircraft. To achieve this we chose a power to weight ratio of 60 W/lb. This translates into a motor that can handle at least 1,320 W. Based on this specification, we chose a brushless outrunner electric motor to power the aircraft. The motor we have chosen is the NTM Prop Drive Series 50-50 580kv. This motor is rated at 2000 W and 580 Kv. Kv is defined as rpm/v, and the lower Kv value of this motor fits our needs since our goal is moderately slow flight. The motor will supply plenty of power at lower rpm, capable of spinning a large, low pitch prop to achieve high thrust as opposed to high speed. The manufacturer supplies experimental thrust data for this motor with different prop configurations. Using a 15x8 propeller, the motor can supply 6.15 kg of thrust or 13.56 lbs. This translates to a thrust to weight ratio of 0.62. Another rule of thumb for radio controlled airplane design is a thrust to weight ratio of 0.33-0.75. This means that this motor and propeller combination should supply sufficient thrust to fly our airplane. If more thrust is needed, the propeller diameter can be increased, which may be necessary due to the large fuselage size. An increase to an 18x10 propeller could provide up to 16.7 lbs of thrust, making the thrust to weight ratio 0.76. With the 2000 W motor and the 15x8 propeller we expect to have sufficient thrust to fly our aircraft as expected. In the immediate future we will be performing bench tests on the motor and propeller combination to determine the actual thrust output. Following our lab tests, we will be performing full systems tests.

2.2.1.4 Fuselage Design The fuselage is the part of the aircraft that connects and houses all of the systems and transmits all of the forces to the aircraft. Our budget was also limited, so we could not spend an excessive amount of money on the fuselage construction. For our aircraft the fuselage was constructed of extruded polystyrene (XPS), commonly known as Styrofoam. The Styrofoam is very rigid but also lightweight, it is also a very protective housing for the electronics. Styrofoam is very easy to work with which makes it very easy to design. The foam can be manipulated by hand or can be machined with a CNC. This, combined with the fact that XPS is cheap and easily obtainable makes it a good choice for the material of the final fuselage. Unfortunately, it would never stand up to the strain of flight on this scale, so we made a carbon fiber skeleton to hold the wing, tail, motor and gimbal; the foam fuselage is easily capable of carrying the remaining payload items.

Figure 3. Lower portion of the fuselage with the carbon fiber skeleton inside

7

Battery

4000 mAh UBEC

Autopilot

Receiver

Servos

2.2.1.5 Flight Circuitry The circuitry on this aircraft plays a large role in its limitations and its capabilities. Proper power management is necessary firstly for the consistent performance of some of the more sensitive instruments and secondly in order to have enough flight time for the aircraft. The electrical components on this plane were broken into two circuits, a flight-critical circuit, and a non-flight critical circuit. The flight critical circuit shown below is the circuit containing the hardware that is essential to guiding the plane to a safe landing should anything go wrong. This circuit is powered by an 11.1 volt lithium phosphate battery with a 4000 mili-Amp-hour capacity. The battery is connected in series to an Ultimate Battery Eliminator Circuit (UBEC) which drops the voltage down to 5 volts and then distributes it out to the autopilot, receiver, and the flight control servos. This circuit is expected to have an average draw of 6 amps and with 4 amp-hours of power we can run this circuit for 40 minutes before we lose power. The second circuit we designed is the non-flight critical circuit. This circuit contains all of the hardware that is not necessary for controlling and landing the aircraft in the event that something goes wrong. A diagram of this circuit is shown below. This circuit is powered by two 22.2 volt lithium phosphate batteries, each with an 8000 mili-Amp-hour capacity, connected in parallel to the Electronic Speed Controller (ESC). The ESC then distributes 22.2 volts to the motor and 5 volts to each of the gimbal servos, gimbal arduino, and the onboard computer we call the pandaboard. This circuit has an expected draw

of 45 amps and with a 16 amp-hour capacity we can expect to run this circuit for 21 minutes. As designed, the non-flight critical circuit is the time limiting factor for flight. We believe that 21 minutes will be sufficient for the flight of this aircraft. It is important to keep in mind that the mission time line has been decreased to 30 minutes and that timer starts as soon as we connect to the receiver. Time will be rolling while we are grounded performing our pre-flight checklist and checking all other wireless connections.

Figure 4. Flow chart of flight critical circuit

2xBattery

8000 mAh

Electronic Speed Controller

Gimbal Servos

Gimbal Arduino

Motor

Pandaboard

Figure 5. Flow chart for non-flight critical circuit

8

2.2.2 Gimbal The primary specification of the gimbal is to be able to point the camera ±60 degrees from nadir in the roll direction to account for targets off of the flight path. It must also be able to traverse angles relatively quickly, and with a fair degree of accuracy. While adding a gimbal to the airframe will allow the onboard camera to be rotated to take pictures of the desired areas, adding the gimbal has the potential to negatively affect the performance of the aircraft. The gimbal and camera assembly will be one of the heaviest components on the aircraft and unlike components like the onboard computer, autopilot and battery that can be mounted within the airframe, the gimbal assembly must be at least partially on the outside of the airframe so that pictures of the ground may be captured. So the gimbal brings challenges of weight and weight distribution along with added drag to the aircraft. There were two major designs to choose from: a yaw-pitch or roll-pitch model.

Figure 6. Yaw-pitch gimbal Figure 7. Roll-pitch gimbal

While both are very similar in general design, there is one key flaw for the yaw-pitch design in relation to our project. As the goal is to take ground photos, the camera will be facing either directly downwards, or near to it for most of the competition. Unfortunately, with a 2-axis gimbal design, gimbal lock occurs at the nadir and zenith points of the outer axis, a fatal flaw in the yaw-pitch design. As such, we focused on and refined the roll-pitch design.

Gimbal Design

Specs Weight % Yaw-Pitch Roll-Pitch

Ease of implementing controls 10% 2 2

Simplicity 15% 3 4

Cost 15% 2 3

Weight 15% 4 3

Low height/Low drag 20% 3 4

Location of gimbal lock 25% 2 4

Total 2.65 3.5

Table 2. Gimbal design decision matrix

9

Pulleys were added to increase the torque provided by the servos as well as increase the angular resolution. Since we had geared up the servos, it was necessitated that they achieve more than 120° of rotation. Our solution was to simply add potentiometers on the other side of the axis, which could then conveniently double as an axle. A somewhat unique feature of the gimbal was to have a mechanical zoom capability as seen in Figure 9.

Figure 8. The detailed design of the gimbal Figure 9. The zoom lens mechanism

2.3 Ground Control Station The Ground Control Station (GCS) gives the team the ability to carry out flight operations and monitor the aircraft during the mission. The GCS software gives the user information about the aircraft’s position and location based on sensor data gathered from the autopilot. The software also has to give the autopilot waypoint locations to navigate towards. Two open source ground station software packages that are designed specifically for the APM 2.5 were considered: APM Mission Planner and QGroundControl. APM Mission Planner was created by the developers at DIYDrones.com, who also designed the APM 2.5. In addition to gathering sensor data and give waypoint information to the autopilot, the APM Mission planner can load any APM firmware directly to the Ardupilot Mega with the easy to use graphical user interface. QGroundControl is better for testing flight missions because the PID gains and the waypoints can be modified in-flight. The interface of QGroundControl is advanced, but the windows can be customized easily to show the all the information needed for the mission. The APM Mission Planner was chosen as the software for the GCS because not only is the interface easy to use, but it also lets the team modify advanced settings and options for the team’s custom aircraft. Some important settings include tuning PID gains, modifying waypoints in-flight and sensor calibration of the APM 2.5.

2.4 Communication Channels Communication between the ground station and aircraft is maintained over three independent channels. These channels were designed and tested to operate over a range of 1 mile.

Channel Frequency Hardware Payload

Data 5.8GHz Ubiquiti Bullet M5 Bulk Data, Gimbal Control

Autopilot 900 MHz 3DR Radio Telemetry, Waypoints

Safety 2.4GHz HobbyKing 6CH TX/RX Autopilot Override Table 3. Communication Summary

10

2.4.1 Data Channel The data channel operates over a 5.8GHz spectrum and is used to transfer bulk imagery data between the aircraft and ground station. This channel is also used to transmit non flight critical telemetry information to any hardware connected to the ground station network. Communication is established with the use of a pair of Ubiquiti Bullet M5 Radios. An L-com HG5819P patch antenna was used on the ground station in order to maintain connection at ranges of over a mile.

2.4.2 Autopilot Channel The autopilot channel operates at 900 MHz and provides a direct and stable connection between the autopilot and the control software on the ground station. This link is used to transmit telemetry information to the ground station as well as waypoint and control updates to the autopilot. The 3DR radio system was selected for this channel primarily for its low cost and long range.

2.4.3 Safety Channel The safety channel is a standard 6 Channel 2.4GHz RC controller. The controller is configured to act as an override for the autopilot’s configured flight path. In the event of a failure or other safety issue the autopilot can be disabled and commands from the controller will directly drive the aircraft servos.

2.5 Flight Hardware The payload camera system is comprised of a Cannon T3i Rebel DSLR and Pandaboard OMAP processor. The DLSR camera is equipped with a 18-55mm zoom lens which is controlled through servo-mechanisms within the gimbal housing. The OMAP processor is the central data hub for the payload and receives IMU and GPS data from the autopilot, along with servo positioning. The camera also controls the camera through a custom C++ application. This application has the ability to control capture parameters, shutter speeds, and specific target image capture based on our computer vision algorithm. Once these images are captured they are then downlinked to the ground station for further image processing. The Pandaboard is equipped with a dual-core ARM processor that outputs 1.2 GHz, 1GB of DDR2 RAM and WIFI connectivity. Open media application processors have a strong open source community, which gave us the ability to program and implement it into our design seamlessly. The EOS t3i DSLR can produce images up to 18 megapixels and can shoot 3.7 frames per second.

Figure 10: DLSR & OMAP

2.6 Flight Software The autopilot must output PWM signals to control servos throughout the aircraft. When the ground station team wants to be able to manually control the aircraft, the autopilot must be able to read signals from the receiver’s channels. The autopilot must also have analog input channels to retrieve data from external sensors, such as an airspeed sensor. Then

11

a transmitter will send the telemetry sensor data to the ground station such as global position (GPS), heading (magnetometer) and orientation (accelerometer, gyroscope) of the aircraft. The team chose the ArduPilot Mega 2.5 (APM 2.5) open source autopilot because it is capable of flying the aircraft fully autonomously, it meets the In-flight Re-tasking objective and it comes standard with the telemetry sensors needed for full autonomy. The APM 2.5 is considerably cheaper and lighter than compared to the Piccolo and Kestrel autopilots that were under consideration.

2.7 Support Software

2.7.1 Mission Planner In addition to the aircraft’s location, altitude, speed and attitude the judges must also be able to see no-fly zones and search areas during the mission. The APM Mission Planner can display search areas, but the software would have to be heavily modified to display no-fly zones. Instead, the team has created a separate program to display no-fly zones along with the GPS location of the aircraft. The GPS location will be gathered from an “http” request over the GCS network server. Error! Reference source not found.Error! Reference source not found. shows an example of the program, rrently in development, displaying a no-fly zone (highlighted in red) and the location of the aircraft at Fairmount Park in Philadelphia, PA. The final program will give the user control of the no-fly zone(s) location and display more information from the aircraft

Figure 11: GCS program to display No-Fly Zone and relative aircraft position.

2.7.1 Gphoto2 Gphoto2 is free software released under the GNU Lesser General Public License. It is cross platform and runs on Linux and other Unix type platforms. Gphoto2 supports more than 1440 cameras and designed for Picture Transfer Protocol, also known as PTP. Gphoto2 also allows for upload and remote controlled configuration and capture.

For our imager system gphoto2 will be responsible for remote image capture while the aircraft is in flight over the search area. Gphoto2 can control capture parameters such as shutter speed, exposure settings, interval settings, and continuous shooting.

Ghoto2 gives us the ability to take complete control over the camera and manually input capture parameters and send real time commands to the camera during flight. We implemented these capture parameters into our C++ as a function. This function gives us the ability to string multiple capture parameters and commands at once. The figure below is a sample of the Gphoto2 function in C++.

12

Figure 12: Gphoto2 C++

This specific example of code is first un-mounting the cameras USB ports giving gphoto2 full control over the camera. The next several lines are naming the directory in which the images will downloaded after capture along with their file name. This also gives us the ability to name images based on GPS location and the time taken. This will be important when sorting through images and determining which images have targets of interest. These targets us interest will then require the camera and gimbal system to point at that specific GPS location and take higher quality pictures. This can all be done during flight of the aircraft and will give our computer vision algorithm higher quality images to analyze. We also have the ability to set exposure and shutter settings in the string commands of the function which will also be crucial to combat the weathers influence on the images taken during flight. All these images are then tagged and targeted for the computer images algorithm to take over and being processing them for targets.

2.7.1 Gimbal Management and Aircraft Status In order to allow for straightforward control of the gimbal system and receive contextual information about the aircraft status a communication management program was written to reside on the onboard computer. This software was based on a simple HTTP Server which will respond to GET requests. The server was written using the Python programming language. The software automatically handles all low level serial communication and provides a JavaScript Object Notation (JSON) response with the information relevant to the request. This software provides a simple and cross-platform method to access all aircraft systems that are important to imagery targeting and capture.

Figure 13 below is the typical response from the aircraft server. While it is out of the scope of this project an advantage of this design is that it would allow for global control of the UAV’s onboard systems given that the ground station is provided with an internet connection. Error! Reference source not found. provides a summary of the commands currently available for access by the ground station. Extending functionality is simple and can be tailored directly to the needs of the ground station software. Once a new URL is reserved all processing is performed by the management software and the ground station receives the exact information requested with no knowledge of the underlying protocols or hardware required to obtain the information.

13

Figure 13. Typical Aircraft Communication Response

Command URL Arguments

Set Gimbal Position /suas/gimbal/degrees/ Pitch, Roll

Get Gimbal Position /suas/gimbal/getposition/ N/A

Set Gimbal Target /suas/gimbal/

Set Camera Zoom /suas/zoom/ Zoom

Get Camera Zoom /suas/getzoom/ N/A

Get Aircraft Location /suas/telemetry/ N/A

Get Aircraft Orientation /suas/gps/ N/A Table 4. Aircraft Server Access Addresses

2.7.2 Image Transfer and Sorting Initially all captured images are saved directly to the aircraft’s onboard computer. This is to prevent problems with the camera control software in the event that the data communication link is lost with the ground station. A second python program runs in the background and monitors the onboard storage directory for file changes. In the event that a new image is detected the status of the data link is verified and the image is routed down to the ground station for further processing. There are various image formats and priorities that can be captured by the camera control software. All information about the image is stored inside a comment tag in the images EXIF data. As with the communication software this data is stored as JSON to allow for simple parsing throughout the software environments in use. The image sorting software parses this information and creates a priority queue to maximize the effectiveness of the available bandwidth between the aircraft and ground station. In the event that there is a loss of communication between the aircraft and ground station the sorting software will halt image transfer and attempt to re-mount the target directory on the ground station. Once a connection has been re-established image transfer will continue as dictated by the priority of the captured images.

14

2. 8 Automatic Detection/Cueing, Classification, and Identification System (ADCCI)

2.8.1 ADCCI Concept Development The ADCCI system is responsible for autonomously performing computer vision analysis to extract glyph information from flight imagery in real-time. Credit is given for varying degrees of reliability of the computer vision algorithms. Drexel’s system intends to fulfill the “classify objective” by identifying two traits aside from position on at least three targets.

OpenCV was chosen as the computer vision library to perform all autonomous imagery analysis. Aside from previous team experience with OpenCV and C++ coding in general, the use of OpenCV allowed for onboard computer vision to take place, which would not be otherwise possible with several other leading computer vision platforms such as Matlab. The ability to perform onboard computer vision allows the opportunity for identifying regions of interest quickly before flying past them. This allows enough time for the gimbal to autonomously zoom and take a detailed picture of potential targets for more reliable in-analysis on the GCS. Onboard, only regions of interest are analyzed and pictures taken. Characteristics such as color and shape are not processed as to decrease computational load and latency. Without onboard processing, there would be high latency associated with streaming images to the GCS and waiting for analysis/reply. Depending on signal and flight conditions, this process could take so long that the plane would have to do another flyby to get a high resolution photo of the picture. The flight path will be planned such that targets will still be manually identifiable without close-up pictures.

2.8.2 ADCCI Embodiment/Detail Design A. Orthorectification: Images are first orthorectified, ensuring target geometry is undistorted and will match

template shapes.

Figure 16. Orthorectified Image

B. Grayscale analysis: Regions of interest are identified based off of the derivatives of several gray scale channels. A given multi-channel image is separated into various grayscale images, which are then analyzed for high contrast edges. Currently, we are analyzing the Saturation channel of HSV, and the U and V channels of YUV. The U and V channels work well with blues and reds, and the saturation channel is a good general purpose grayscale image for detecting contrast.

Figure 17. Sample Grayscale Images

15

C. SIFT Feature Detection: Originally, SIFT feature detection was tested on both sample imagery from past competitors as well as simulated indoor imagery. This method proved unreliable due to the lack of information associated with the real-world imagery. Furthermore, As SIFT is intended for feature-rich environments with complex geometry and all shapes being analyzed were very simple, false positives were very common.

D. Edge Detection: Instead, a simpler, more traditional approach has been taken. As stated above, the gradients of various grayscale images are taken. Canny edge detector is then used to find high contrast edges. Thresholding is based off of many image properties ranging from weather conditions to vibration levels to image blurriness. Autothresholding was not used to reduce the possibility of false positives in target-void imagery. Rather, expected contrast levels can be anticipated and thresholded accordingly.

E. Logic Filters: As seen above, Canny edge

detects the shape contours very well, but can also detects any other high contrast edges such as roads, cracks, etc. based on thresholding and terrain properties. In order to filter out any remaining false positives, logic filters have been put in place. These logic filters currently consist of area, aspect ratio, and overlap

detectors. If a potential target’s area is outside of a given range based on pixel density, the contour will be erased. Furthermore, if the aspect ratio is very large, i.e. in the case of road lines, it will also be removed. Aspect ratio is automatically set up to measure along the longest axis of a given contour. Lastly, overlap detectors ensure only the outer contour of a given target is kept in case the alphanumeric is also segmented.

Figure 15 Canny Edge Detector Output Contours

Figure 14 SIFT feature detector and Bruteforce Matcher in combination, matching incorrectly

Figure 16. Results from Logic Filtering

16

F. Color detection: At this point, all targets have hopefully been found and false positives filtered out. It should be noted that onboard processing stops here, whereas GCS processing continues on to identify glyph traits. In order to determine background color, a histogram is used. Because the image has already been segmented by the contour, the histogram of the glyphs will have a large peak representing the color of the background. This peak is located by its H, S, and V value. The S and V values are thresholded to determine if the color is black or white. If not, the image is classified according to its hue into various bins corresponding to colors such as blue, green, orange, etc.

G. Contour Detection: Once color is determined, the background shape can be analyzed. This is done by comparing the contour to a series of preloaded shapes expected to be seen in the competition. Each template is first scaled to the approximate size as the contour. Next, it is compared to all template shapes and rotated in small degree increments. Whatever shape and rotation has the lowest percent overlap from a simple binary xor comparison is considered the correct shape. Although surprisingly simple, this method has proved far more reliable in competition settings than SIFT, SURF, and Hu moment comparison. It works extremely well (relatively speaking) with blurry, noisy, low data imagery.

H. Other character traits: Drexel’s system does not currently perform any alphanumeric analysis. If time allows, color detection may be implemented as it is simple and could provide a second glyph characteristic if either of the background traits are incorrect. Because the only level above characterization is identification, which requires 100% accuracy, it was deemed “not worth the trouble” as a first year team to strive for such a high-fidelity goal with a newborn system.

Figure 17. Example histogram of green triangle. X axis represents hue, Y axis represents saturation

Figure 18. Shape Comparison Example

17

2.8.3 Image Stitching: Image stitching has still not been implemented, but will be if time allows. Conceptually, a latitude/longitude calibrated base image could be loaded in memory for the stitching program. As targets are found, their images can be stitched to the calibration base image. The new coordinates and distortion of the target image can be used to interpolate the exact location of the image within several pixels, correlating to an extremely precise location easily within the 50 foot objective. OpenCV provides image stitching support, but has not been tested yet.

2.9 Mission Planning Key personnel to mission performance include the supervisor/pilot, flight software operator, computer vision technician, antenna pointer, and manual image taggers. The supervisor will oversee operations, takeoff/land the plane, perform manual safety override if necessary, and interface with the judge. The flight software operator will interface with the aircraft through the Mission Planner software and make any in-flight adjustments necessary. The computer vision technician will observe target recognition and ensure its proper operation. The antenna pointer will keep the antenna manually aimed that the aircraft. Manual image taggers will operate several computers in parallel to search and manually tag glyphs not classified by the ADCCI System.

Mission planning revolves heavily around APM’s Mission Planner software, designed to interface with the autopilot and update tasks real-time. All navigation will be waypoint based. Waypoints will be preloaded and can be added dynamically mid-flight. For the search area, Mission Planner offers capabilities to generate a flight path of waypoints based on a fenced area, or search area in this case. In both cases, pictures will be taken based off of distance intervals along the flight path. Takeoff and landing will be performed manually, and SRIC will be ignored.

3. System Evaluation

3.1 Simulation Initially, system testing was performed through the use of Hardware in the Loop (HIL) simulation. The Xplane simulation software acted as the substitute for the physical aircraft. The Ardupilot 2.5 autopilot is capable of directly

Figure 19. Results of OpenCV Processing

18

interfacing with Xplane in order to perform the HIL simulation. This configuration allowed the team to test the performance of various subsystems such as gimbal targeting and no-fly zone display before a flight capable platform was available. The communication protocol between the simulator and autopilot was found to be a modified version of the typical communication with the ground station and only minor modifications to existing software had to be made in order to parse the simulator communication.

3.2 Bench Testing A multi-rotor platform was available for initial flight testing of the autopilot system, however as the complexity of the payload grew it became unfeasible to fly with the available airframe. This resulted in a significant amount of system testing performed in a laboratory environment. Network communication and image capture and processing were tested using a prototype ground station setup and a series of hoists and simulated targets.

Figure 20. Simulated Ground Targets

Range testing was performed in a rural environment in order to best simulate the expected conditions for the competition. Figure 21 shows the range tests results for the 900 MHz autopilot link. System range was found to be approximately 1 mile, satisfactory for our requirements.

Figure 21. Range Testing Results

3.3 Flight Testing Unfortunately as a result of unforeseen hardware failure and the size of the system payloads it has not been possible to perform flight testing of the full system. The performance of the autopilot system has however been established through a series of flight tests at fields surrounding the university campus.

19

4. Safety The safety of the operator and all other by-standers was held in the highest regard throughout the design and testing of

this system. Hazardous materials were avoided as much as possible in the construction of our aircraft as well. The most

hazardous material we have on the plane is in the lithium polymer batteries we are using the power the aircraft.

Because these batteries can be very dangerous after being battered in a crash we wrapped all of them in plastic bags

before they were installed. We expect the bags to contain most minor leakages of caustic agents that are inside the

battery.

On the manufacturing and design level it was necessary that we ensured the aircraft could take off, fly, and land without

losing pieces or falling apart. From a safety stand point we wanted to prevent the aircraft from breaking up and

dropping pieces of the structure on pedestrians. With a motor as powerful as ours connected directly to the rigid

skeleton of the aircraft it is a foreseeable problem that the vibrations could loosen hand-tightened screws. To prevent

the hardware from separating we used loctite and lock washers on all nuts and screws, structural or non-structural, in

the gimbal and in the fuselage assemblies. 5-minute epoxy and other non-water-soluble liquid adhesives were also

generously used to fasten and secure components that could not be bolted together.

An additional concern was that the aircraft could fly uncontrollably into a populated roadway and cause an accident. To

minimize the risk of this happening, an extensive pre-flight checklist was created and all pilots and co-pilots were

instructed to follow the list during every test flight of the competition aircraft and prototype aircrafts. The goal of the

pre-flight checklist was to be able to confirm before takeoff, that in flight the aircraft would handle as expected. All flight

control surfaces were trimmed and then checked for proper actuation while the aircraft was still on the ground. A motor

test was always conducted and the mechanical linkages between the components of the aircraft were physically tested

as well. A pilot’s skill is another big factor of flight safety.

Before any team member is cleared to fly a plane in a public space he is required to pass a series of tests. The goal of the

tests is to fully prepare the pilot for any real-world scenarios that might occur while flying our prototypes in the field,

and to minimize any risk of a costly catastrophic crash of our airplane. A flight simulator called ClearViewSE was used to

give pilots their first training on the computer. In this program the pilot had to complete 4 tasks before he could move to

a physical RC plane. The tasks are to log a minimum of 3 hours of flight time in the simulator, perform a figure 8

maneuver below tree level without crashing, land the plane safely after instant power loss to the engine at any time,

and safely perform a powered landing. Once these four criteria are successfully and satisfactorily accomplished, the next

phase of pilot training is a repetition of these 4 tasks with a 5-foot trainer plane we bought for this purpose. With all of

these safety features in place we believe we have given ourselves the best possible chance and designing, testing, and

implementing our system with the highest level of safety and the highest level of compliance with all AMA safety

guidelines.

20

5. References 1. Ulaby, F.T., Fundamentals of Applied Electromagnetics. 6th ed. 2010, New York. 2. Lathi, B.P., Modern Digital and Analog Commnunication Systems. 4th ed. 2009, New

York: Oxford University Press. 3. Semtech, FCC Regulations for ISM Band Devices. 4. Smith, S., Microelectronic Circuits. 6th ed. 2010, New York: Oxford University. 5. Sadraey, Mohammad, 2012, Aircraft Design: A Systems Engineering Approach, Wiley

6. Freeman, R.E., 1984, Strategic Management: A Stakeholder Approach, Pitman, Boston

7. http://www.ef-uk.net/data/wcl.htm 8. http://www.boeing.com/commercial/707family/product.html 9. http://adamone.rchomepage.com/design.htm#calculate 10. http://www.modelaircraft.org/files/105.PDF