how to build & use opencl on opencv & android ndk
DESCRIPTION
This manual is “How to Build” manual for OpenCV with OpenCL for Android. If you want to “Use OpenCL on OpenCV” ONLY, Please see http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSampleTRANSCRIPT
©SIProp Project, 2006-2008 1
How to Build & Use OpenCLon OpenCV & Android NDK
Noritsuna Imamura
©SIProp Project, 2006-2008 2
!!!!!!ATTENTION!!!!!!
This manual is “How to Build” manual.
If you want to “Use OpenCL on OpenCV” ONLY,
Please see
http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample
©SIProp Project, 2006-2008 3
How to Build OpenCL on OpenCV
for Android NDK
©SIProp Project, 2006-2008 4
Download OpenCV & NDK tool
› wget https://github.com/Itseez/opencv/archive/2.4.10.1tar.gz
› tar zxfv 2.4.10.1.tar.gz
› mkdir ~/android/› cd ~/android/› wget http://dl.google.com/android/android-
sdk_r23.0.2-linux.tgz› wget
http://dl.google.com/android/ndk/android-ndk-r10c-linux-x86_64.bin
› tar zxfv android-sdk_r23.0.2-linux.tgz› ./android-ndk-r10c-linux-x86_64.bin
©SIProp Project, 2006-2008 5
Edit Setup File
./opencv-2.4.10.1/platforms/scripts/cmake_android_arm.sh
Add WITH_EIGEN=off option.Currently version has some bugs.
› cmake -DWITH_EIGEN=OFF -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.cmake $@ ../..
©SIProp Project, 2006-2008 6
Set Path to SDK/NDK
› export ANDROID_NDK=~/android/android-ndk-r10c
› export ANDROID_SDK=~/android/android-sdk-linux
©SIProp Project, 2006-2008 7
Add “SO loading“ func for Android 1/3
Change “SO loading“ func for Linux to this funcwithout Android.
~/opencv-2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp
› #if defined(__linux__)↓
› #if defined(__linux__) && !defined(__ANDROID__)
©SIProp Project, 2006-2008 8
Add “SO loading“ func for Android 2/3
Add “SO loading“ func for Android. (continue to next page)
~/opencv-2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp
› #if defined(__ANDROID__)› #include <dlfcn.h>› #include <sys/stat.h>
› #if defined(__ARM_ARCH_8A__) || defined(_X64_)› static const char *default_so_paths[] = {› "/system/lib64/libOpenCL.so",› "/system/vendor/lib64/libOpenCL.so",› "/system/vendor/lib64/egl/libGLES_mali.so"› };› #else› static const char *default_so_paths[] = {› "/system/lib/libOpenCL.so",› "/system/vendor/lib/libOpenCL.so",› "/system/vendor/lib/egl/libGLES_mali.so"› };› #endif
›
©SIProp Project, 2006-2008 9
Add “SO loading“ func for Android 3/3
› static int access_file(const char *filename)› {› struct stat buffer;› return (stat(filename, &buffer) == 0);› }
› static void* GetProcAddress (const char* name)› {› static void* h = NULL;› unsigned int i;› if (!h)› {› const char* name;› for(i=0; i<(sizeof(default_so_paths)/sizeof(char*)); i++)› {› if(access_file(default_so_paths[i])) {› name = (char *)default_so_paths[i];› h = dlopen(name, RTLD_LAZY);› if (h) break;› }› }› if (!h)› return NULL;› }
› return dlsym(h, name);› }› #define CV_CL_GET_PROC_ADDRESS(name) GetProcAddress(name)› #endif
©SIProp Project, 2006-2008 10
Execute Auto Config & make
This Script make Makefile for OpenCV.
In “~/opencv-2.4.10.1/platforms/scripts/build_android_arm.sh”
Make !!!
› cd ~/opencv-2.4.10.1/platforms/scripts/› ./cmake_android_arm.sh
› cd ~/opencv-2.4.10.1/platforms/scripts/build_android_arm/
› make
©SIProp Project, 2006-2008 11
Where are Made Files?
OpenCV Libs.
In “~/opencv-2.4.10.1/platforms/build_android_arm/lib”
highgui
core
etc...
3rd Party Libs for OpenCV.
In “~/opencv-2.4.10.1/platforms/build_android_arm/3rdparty/lib”
libIlmImf
libjasper
libjpeg
libpng
libtiff
©SIProp Project, 2006-2008 12
Copy libs to sample program
OpenCV Libs.
cp ~/opencv-2.4.10.1/platforms/build_android_arm/lib/armeabi-v7a/* ~/opencv-2.4.10.1/samples/android/native-activity/libs_opencv/
3rd Party Libs for OpenCV.
cp ~/opencv-2.4.10.1/platforms/build_android_arm/3rdparty/lib/armeabi-v7a/* ~/opencv-2.4.10.1/samples/android/native-activity/libs_opencv/
©SIProp Project, 2006-2008 13
How to Use OpenCL on OpenCV
for Android NDK(Native App)
©SIProp Project, 2006-2008 14
Download Sample Source Code
http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample
Based OpenCV-2.4.10.1 & Android 4.4
Made Android Native Activity.
Convert RGB Color Image to Black&White Image.
©SIProp Project, 2006-2008 15
Native Application
NDK wo/ADT
Standard Android Application for C/C++
Only C/C++ on LimitedLibrary Layer
Advantage
Only C/C++
DirectCall C/C++ API
Dis-Advantage
Use a few Android Tools
A few Docs from Google Developer Site & Blogs
APK File(Your Application)(C/C++)
Library Layer (C/C++)
Kernel/Driver Layer(C/ASM)
Call as C/C++ APIs
Call as SysCall(C/ASM)
Call Stack
©SIProp Project, 2006-2008 16
How to Make this Sample Source Code from OpenCV’s Sample Project
©SIProp Project, 2006-2008 17
Use NativeActivity of OpenCV
This Sample uses Only C/C++ Code.
~/opencv-2.4.10.1/samples/android/native-activity
Structure of NativeActivity
AndroidManifest.xml <- Property File for Android
res <- Directory that has Resource file(Image,Voice)
jni <- Directory that has files for NDKAndroid.mk <- make file of NDK
Application.mk <- property file of NDK
native.cpp <- Sample App
Other Files/Dir, don’t use.
©SIProp Project, 2006-2008 18
Edit AndroidManifest.xml
Replace “application” Section
1. <application android:label="@string/app_name" android:hasCode="false">
2. <activity android:name="android.app.NativeActivity"3. android:label="@string/app_name"4.
android:configChanges="orientation|keyboardHidden">5. <!-- Tell NativeActivity the name of or .so -->6. <meta-data android:name="android.app.lib_name"7. android:value="native_activity" />8. <intent-filter>9. <action
android:name="android.intent.action.MAIN" />10. <category
android:name="android.intent.category.LAUNCHER" />11. </intent-filter>12. </activity>13. </application>
©SIProp Project, 2006-2008 19
About AndroidManifest.xml
API Level
Permission
1. <!-- This is the platform API where NativeActivity was introduced. -->
2. <uses-sdk android:minSdkVersion=“9" />
1. <uses-permission android:name="android.permission.CAMERA"/>
©SIProp Project, 2006-2008 20
Create jni/Android.mk 1/9
1. Delete jni/Android.mk & Create the new jni/Android.mk
2. Set LOCAL_PATH & include
1. LOCAL_PATH := $(call my-dir)
2. include $(CLEAR_VARS)
©SIProp Project, 2006-2008 21
Create jni/Android.mk 2/9
1. Set OpenCV’s Modules & Libs
1. OPENCV_MODULES:=contrib legacy ml stitching superres objdetect ts videostabvideo photo calib3d features2d highguiimgproc flann ocl core androidcamera
2. OPENCV_3RDPARTY_COMPONENTS:=assetmanager tbb libjpeg libpng libtiff libjasperIlmImf
3. OPENCV_CAMERA_MODULES:= native_camera_r2.2.0 native_camera_r2.3.3 native_camera_r3.0.1 native_camera_r4.0.0 native_camera_r4.0.3 native_camera_r4.1.1 native_camera_r4.2.0 native_camera_r4.3.0 native_camera_r4.4.0
©SIProp Project, 2006-2008 22
Create jni/Android.mk 3/9
Set Loading OpenCV Libs Function
1. $(foreachmodule,$(OPENCV_MODULES),$(eval $(call add_opencv_module,$(module))))
1. define add_opencv_module2. include $(CLEAR_VARS)3. LOCAL_MODULE:=opencv_$14.
LOCAL_SRC_FILES:=../libs_opencv/libopencv_$1.a
5. include $(PREBUILT_STATIC_LIBRARY)6. endef
©SIProp Project, 2006-2008 23
Create jni/Android.mk 4/9
Set Loading OpenCV Libs Function
1. $(foreachmodule,$(OPENCV_3RDPARTY_COMPONENTS),$(eval $(call add_opencv_3rdparty_component,$(module))))
1. define add_opencv_3rdparty_component2. include $(CLEAR_VARS)3. LOCAL_MODULE:=$14.
LOCAL_SRC_FILES:=../libs_opencv/lib$1.a5. include $(PREBUILT_STATIC_LIBRARY)6. endef
©SIProp Project, 2006-2008 24
Create jni/Android.mk 5/9
Set Loading AndroidCamera Libs Function
1. $(foreachmodule,$(OPENCV_CAMERA_MODULES),$(eval $(call add_opencv_camera_module,$(module))))
1. define add_opencv_camera_module2. include $(CLEAR_VARS)3. LOCAL_MODULE:=$14.
LOCAL_SRC_FILES:=../libs_opencv/lib$1.so5. include $(PREBUILT_SHARED_LIBRARY)6. endef
©SIProp Project, 2006-2008 25
Create jni/Android.mk 6/9
Set loading info lib
1. include $(CLEAR_VARS)2. LOCAL_MODULE:=opencv_info3. LOCAL_SRC_FILES:=../libs_opencv/libopenc
v_info.so4. include $(PREBUILT_SHARED_LIBRARY)
©SIProp Project, 2006-2008 26
Create jni/Android.mk 7/9
Set Include Dir
Set Module name & Source Code
OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp
1. include $(CLEAR_VARS)
2. LOCAL_C_INCLUDES+=./jni/include
3. LOCAL_MODULE := OpenCV_CL4. LOCAL_SRC_FILES :=
OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp
©SIProp Project, 2006-2008 27
Create jni/Android.mk 8/9
Set Loading Lib & OpenCV Lib
LOCAL_LDLIBS-lm -llog -lc -ldl -lz -landroid
LOCAL_STATIC_LIBRARIES$(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS)
1. LOCAL_LDLIBS += -lm -llog -lc -ldl -lz -landroid
2. LOCAL_STATIC_LIBRARIES := android_native_app_glue $(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS)
©SIProp Project, 2006-2008 28
Create jni/Android.mk 9/9
Set Library Type
Make SO libBUILD_SHARED_LIBRARY
Set Loading Android module
Use “native_app_glue” module$(call import-module,android/native_app_glue)
1. include $(BUILD_SHARED_LIBRARY)
2. $(call import-module,android/native_app_glue)
©SIProp Project, 2006-2008 29
Setup/Make Build.xml 1/2
Making Setting File for Android using ANT
Today, Development on CUI Only!!!Don’t use Eclipse!
› android list target› Available Android targets:› ----------› id: 1 or "android-9"› Name: Android 2.3.1› Type: Platform› API level: 9› Revision: 2› Skins: WQVGA400, QVGA, HVGA, WVGA800 (default), WVGA854› ABIs : armeabi› ----------› id: 2 or "android-18"› Name: Android 4.3› Type: Platform› API level: 18› Revision: 2› Skins: WXGA800, WQVGA400, QVGA, WQVGA432, WXGA800-7in, WVGA800
(default), HVGA, WXGA720, WSVGA, WVGA854› ABIs : armeabi-v7a› ----------
©SIProp Project, 2006-2008 30
Setup/Make Build.xml 2/2
Make Build.xml
› android update project --path . --target 2
©SIProp Project, 2006-2008 31
Build NativeActivity
Build C/C++ Source Codes
Make APK File
› ndk-build [clean all]
› ant [debug|release]
©SIProp Project, 2006-2008 32
Install NativeActivity.apk
Install APK File
UnInstall APK File
Show Log
› adb install bin/NativeActivity-debug.apk
› adb logcat
› adb uninstall com.example.native_activity
©SIProp Project, 2006-2008 33
adb(Android Debug Bridge)
Functions
Shell on Android Deviceadb shell
Show Connected Android Devicesadb devices
Install/Uninstall APK Fileadb install/uninstall [App Name]/[Package Name]
Send/Get File to/from Android Deviceadb push/pull [from File Path] [to File Path]
Show Log (DDMS)adb logcat
©SIProp Project, 2006-2008 34
Quick Explanation to OpenCL
1. void android_main(android_app* app) {2. (snip)3. cv::Mat drawing_frame;4. (snip)
5. colorSampleImage = imread(file_path);6. (snip)
7. // Create Mat Object for OpenCL8. cv::ocl::oclMat oclIn(colorSampleImage), oclOut;9. // Change Color RGB->Black&White on OpenCL10. cv::ocl::cvtColor(oclIn, oclOut, cv::COLOR_BGR2GRAY);11. // Download Mat Object for OpenCL to Mat for CPU12. oclOut.download(drawing_frame); 13.14. engine_draw_frame(&engine, drawing_frame);
©SIProp Project, 2006-2008 35
How to Set up OpenCL Driver
adreno-3xx: Qualcomm's Adreno 3xx GPUs
mali-t6xx: ARM's Mali-T6xx GPUs
©SIProp Project, 2006-2008 36
Download OpenCL Drivers
OpenCL Drivers was deleted since 4.2
adreno-3xx: Qualcomm's Adreno 3xx GPUs
mali-t6xx: ARM's Mali-T6xx GPUs
Download from:
https://github.com/madeye/opencl-android-proprietary
©SIProp Project, 2006-2008 37
Set up Drivers to your Android
Ex. adreno-3xx: Qualcomm's Adreno 3xx GPUs
1. $ adb push kitkat/system/vendor/lib /sdcard/2. $ adb shell3. $ su4. # mount -o rw,remount -t yaffs2 /dev/block/mtdblock3
/system5. # cp /sdcard/libOpenCL.so /system/vendor/lib/6. # cp /sdcard/libllvm-a3xx.so /system/vendor/lib/7. # chmod 644 /system/vendor/lib/libOpenCL.so8. # chmod 644 /system/vendor/lib/libllvm-a3xx.so