1

Qualcomm® Hexagon™ SDK Optimize Your Multimedia Solutions

Jack May, Senior Staff Engineer Qualcomm Technologies, Inc.

Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.

3

What is the Hexagon SDK?

4

Designed to easily utilize the capabilities of the Hexagon DSP

Uses the ability to dynamically load and execute code on the Hexagon DSP

Helps reduce development effort and promotes a rapid development cycle

Applications include − General compute

− Audio

− Voice

− Imaging

− Computer vision

Overview What is the Hexagon SDK?

5

Provide customization of the Hexagon DSP without rebuilding the static image − Remote procedure call mechanism between the HLOS and the Hexagon DSP

− Dynamically loadable objects loaded at runtime.

− Dynamic module interfaces to the Audio/Voice/.. Frameworks

Provide multiple debugging options − USB based debugger

− Diagnostic logging

Provide unit and system level test frameworks and capabilities

Provide both in-simulation and on-target environments

Goals What is the Hexagon SDK?

6

Windows installer

Installs the SDK components (docs, examples, tools, …)

Results in a complete, self-contained development environment

Minimizes environment changes that may impact other tools or installations

Multiple SDK versions may be installed side by side

Downloads 3rd party dependencies

No global PATH or environment variable modifications

Dependencies are self-contained and local to the SDK − Except

− Hexagon Tools

− Python

Linux support on the roadmap

Installation What is the Hexagon SDK?

7

What is in the Hexagon SDK?

8

Fast Remote Procedure Calls (FastRPC)

Dynamic loading

Remote debugger

Eclipse plugin

Hardware development platforms

Optimized Hexagon libraries

Real-time Operation System (RTOS)

Comprehensive tool chain

Diagnostic logging

Notable features What is in the Hexagon SDK?

9

Remote procedure calls between the applications processor and the Hexagon DSP

Fast − Synchronous

− Zero copy

− Cache coherent

Functions called on the Hexagon DSP may reside in dynamically loaded code

Maintains cache coherency for input and output buffers

Calling HLOS threads handled by corresponding threads on the Hexagon DSP

Threads are automatically cleaned up when HLOS process exits

Notable features: FastRPC What is in the Hexagon SDK?

10

DSP Applications processor

Notable features: FastRPC What is in the Hexagon SDK?

In-process

Transport

Application calculator library

calculator_skel

calculator_sum()

#include "calculator.h" int calculator_sum( const int* vec, int vecLen, int64* res) { int ii = 0; *res = 0; for(ii = 0; ii < vecLen; ++ii) { *res = *res + vec[ii]; } return 0; }

calculator_sum()

calculator_stub

calculator_sum()

Sample code subject to SDK license agreement. Available at developer.qualcomm.com

11

Notable features: FastRPC: Performance What is in the Hexagon SDK?

Size of buffer 8074 in (us) 8074 out (us)

0 62

32 kB 80 90

64 kB 84 94

128 kB 85 106

1 MB 177 211

4 MB 261 212

8 MB 377 303

16 MB 612 538

− Measured with Application and DSP processor clocks at max − 8074 DragonBoard™ from Intrinsyc

− No change in data and DSP returns immediately

− These are minimum times, and does not include RAM to cache sync times (as if prefetched) Source: Qualcomm Technologies, Inc. internal testing

12

Allows for the addition of code/data into the Hexagon DSP at runtime

Faster development cycles, only have to build your shared object

Code/data is built into shared object files (.so) − Executable and Linkable Format (ELF)

− Stored on the HLOS’s file system and read by the Hexagon DSP when loaded

Functions and variables become accessible to the static DSP image once the shared object is loaded

Native support for shared objects in the audio and voice frameworks

Notable features: Dynamic Loading What is in the Hexagon SDK?

13

lib.so

Fxn()

Notable features: Dynamic Loading What is in the Hexagon SDK?

HLOS File System

lib.so

Fxn()

aDSP Static Image

h = dlopen(“lib.so”);

pFxn = dlsym(h, “Fxn”);

pFxn();

dlclose(h);

lib.so

Fxn()

14

Easier to create, build, run, test, and debug Hexagon modules

Project templates

Unit and integration testing

Simulator and on-target debugging

Syntax aware and highlighting for Hexagon assembly, C/C++, and IDL

Custom views − Connected devices

− Profiler

− Resource analyzer

− Audio module database

− Logging

Notable features: Eclipse IDE plugin What is in the Hexagon SDK?

15

USB based software, no hardware JTAG required

Industry standard debug tools (GDB with LLDB on roadmap)

Integrated with the Hexagon SDK Eclipse plugin

Single or all thread stop debugging

Simulator and on-target debugging

Notable features: Remote debugger What is in the Hexagon SDK?

16

Quick development cycle from picking up the Hexagon SDK to running your code on-target

Affordable hardware based development platforms provided by Intrinsyc

Multiple form factors supported − Exposed boards (DragonBoards)

− Tablets (MDPs)

Hardware support provided by Intrinsyc − http://www.intrinsyc.com/products/qualcomm/dragonboard-development-kits.aspx

Notable features: Hardware development platforms What is in the Hexagon SDK?

17

C and Assembly optimized libraries for common operations − FIR, IIR

− FFT, IFFT, FHT

− Up sampler

− Computer vision (FastCV) − feature/object detection

− motion & object tracking

− color conversion

− 3D reconstruction

− image processing

− Lots more …

Notable features: Optimized Hexagon libraries What is in the Hexagon SDK?

18

QuRT™ software: real-time operating system for the Hexagon processor

Low overhead both in memory and processing requirements

Supports − Priority-based preemptive multithreading

− Thread synchronization and communication (mutex, signal, semaphore, barriers, pipes, …)

− Timers

− Interrupts

− Exceptions

− Memory management

Notable features: RTOS What is in the Hexagon SDK?

QuRT is a product of Qualcomm Technologies, Inc.

19

ISO C/C++ compiler Assembler Standard (libc) and processor specific libraries Intrinsics Linker and archiver Debugger (gdb) Profiler (gprof) Misc utilities − ELF viewer − nm − strings − size

Notable features: Comprehensive tool chain What is in the Hexagon SDK?

20

Runtime diagnostic logging

Viewable in logcat or on host (PC)

Integrated with the Hexagon SDK Eclipse plugin

Supported in simulation and on-target

Notable features: Diagnostic logging What is in the Hexagon SDK?

21

What can you do with the Hexagon SDK?

22

Offload compute tasks from the application processor onto the Hexagon DSP − Lower power − Frees up resources on the application processor − Appears to caller as if calling a local library

Easier to do − Define interface in IDL − Use existing C code or optimize with Hexagon DSP assembly − Build using the Hexagon Tools suite − Call via the stub library on the application processor.

Moving data between processors is transparent to callers and implementers − Cache coherent − Zero copy

Calculator example in the Hexagon SDK

General compute What can you do with the Hexagon SDK?

23

Add custom algorithms, encoders, and decoders to the audio framework

Dynamically load and unload audio use cases on an as-needed basis − Saves memory

Communicate with your audio module at runtime − Tuning

− Feature enabling/disabling

− User controlled settings

Clearly defined interface to implement (APPI or CAPI) − Unit test frameworks to validate that your module is ready to run on the Hexagon DSP

Lots of APPI and CAPI examples in the Hexagon SDK

Audio What can you do with the Hexagon SDK?

24

Add custom processing algorithms to the voice framework

Communicate with your voice module at runtime − tuning

− Feature enabling/disabling

− User controlled settings

Clearly defined interface to implement (CAPI_V2) − Unit test frameworks to validate that your module is ready to run on the Hexagon DSP

CAPI_V2 passthru and gain examples in the Hexagon SDK

Voice What can you do with the Hexagon SDK?

25

Library of frequently used functions optimized to run on mobile devices − Math / vector operations

− Image processing and transformation

− Feature and object detection

− 3D reconstruction

− Color conversion

− Clustering and search

− Motion and object tracking

− Share and drawing

− Memory management

downscaleBy2 and cornerapp examples in the Hexagon SDK

Computer vision What can you do with the Hexagon SDK?

26

Demo

27

developer.qualcomm.com

Augmented reality

Context awareness

Peer to peer

Wireless health

Graphics & gaming

Computer vision

Android applications

App development SDKs & tools

Marketing opportunities

Case Studies & Tutorials

News & Information

Hardware development platforms

Forums & technical support

Tools & resources to help developers build, integrate, and optimize

Qualcomm Developer Network

28

For more information on Qualcomm, visit us at: www.qualcomm.com & www.qualcomm.com/blog

Thank you

FOLLOW US ON:

© 2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All rights reserved. Qualcomm, DragonBoard, and Hexagon are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq and QuRT are trademarks of Qualcomm Incorporated. All trademarks of Qualcomm Incorporated are used with permission. Other product and brand names may be trademarks or registered trademarks of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.

29

Additional Slides / APPENDIX

30

Notable features: Eclipse IDE plugin: On-target debug What is the Hexagon SDK?

Source: Hexagon SDK

31

Notable features: Eclipse IDE plugin: Compute off-load What is the Hexagon SDK?

Source: Hexagon SDK

32

Notable features: Eclipse IDE plugin: Custom views What is the Hexagon SDK?

AMDB View Identifies the devices (including hot plug), provides the facility to add/remove/configure the audio modules

Quick Profile Viewer Displays static/dynamic profiling information for a project,

Device Viewer Displays the connected devices and their state,

Source: Hexagon SDK