OpenCV4Android
前文曾详细探讨了关于OpenCV的使用,原本以为天下已太平。但不断有人反应依然配不好OpenCV4Android,不能得心应手的在Android上使用OpenCV,大量的精力都浪费在摸索配置上。尤其是OpenCVManager诞生之后,更让人无语,大家第一个反应就是如何才能不安装OpenCVManager,因为要多安装这个东西对客户来说体验太不好了。咱家昨夜研究至两点,今早七点起床,终于把头绪理清了。下面咱家以之前做过的一个基于OpenCV2.3.1,android通过jni调用opencv实现人脸检测的实例来逐个回答,如何在Android上使用Java接口而不安装OpenCVManager,及通过jni方式使用OpenCV的三种方式。
先来看JNI调OpenCV的三种方式。很多人会吃惊肿么JNI调OpenCV还会有3种方式,长久以来大量网上教程都说在Android上只有Java和JNI两种方式使用OpenCV,怎么又冒出来3种使用JNI的方式。经本人研究,确实有3种调JNI的方式,就连官网指导文档都模棱两可,所以让很多人不知所措。这三种方式分别是:
1、使用静态的OpenCV库的方式;
2、使用动态的OpenCV库的方式;
3、同时使用Java的API又使用JNI的接口的方式,此时编译时一般使用的是动态链接OpenCV库的方式。
要说明的是,这三种方式均无需安装OpenCVManager,区别在于mk文件的不同。个人最推崇的就是第一种方式,第一种方式也是和OpenCV2.3.1在JNI调OpenCV使用完全吻合的一种方式。本文是以windows平台最新的OpenCV-2.4.9-android-sdk为基础,使用2.4.9的OpenCV4Android需要使用NDK版本为r9,本人使用的是android-ndk-r9d的版本。之所以昨晚捣腾到2点,就是因为之前使用的ndk r7的版本,怎么编都编不过,因少东西报上千行错误。android-ndk-r9d安装十分简单,只需要解压缩配置一个环境变量即可。
一、Android以JNI调OpenCV的第一种配置方法:
Application.mk文件里的内容如下:
APP_STL:=gnustl_static
APP_CPPFLAGS:=-frtti -fexceptions
APP_ABI:= armeabi-v7a
这三种方式的Application.mk都一样,所以往后不说了。在Application.mk里还可以配置APP_PLATFORM=17类似这种,当然不配置完全可以。
Android.mk内容如下:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
OpenCV_INSTALL_MODULES:=on
OPENCV_CAMERA_MODULES:=off
OPENCV_LIB_TYPE:=STATIC
ifeq ("$(wildcard $(OPENCV_MK_PATH))","")
include D:\ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\OpenCV.mk
else
include $(OPENCV_MK_PATH)
endif
LOCAL_MODULE := ProcessImg
LOCAL_SRC_FILES := DetectFace_JNI.cpp \
src/copyToAssets.cpp \
src/detectFace.cpp
LOCAL_LDLIBS += -lm -llog
include $(BUILD_SHARED_LIBRARY)
逐一解释下,OpenCV_INSTALL_MODULES:=on的意思是自动将依赖的OpenCV的so库拷贝到libs目录下,但很遗憾的是,这个命令只对OPENCV_CAMERA_MODULES有效。只有当OPENCV_CAMERA_MODULES:=on时,可以看到他会自动将里面的带camera的so拷贝至工程下的libs文件夹下。include D:\ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\OpenCV.mk这句话比较关键,这是我安装OpenCV-2.4.9-android-sdk的地方,我将其安装到了D盘。而我的工作空间在E盘也是ok的。而不用像OpenCV2.3.1使用时,限制这个解压缩包的位置了。LOCAL_MODULE 是要生成的库的名字,LOCAL_SRC_FILES 是jni文件夹下的cpp文件,其中的src说明我的jni下还有个子文件夹名字是“src”,这块替换成自己的源码文件就ok了。
为了测试的严谨性,在工程里将libs文件夹的东西,和obj文件夹下的东西全删了。用cygwin进到工程,输入ndk-build,看到如下信息:
<span style="font-family: 'Comic Sans MS';"><span style="font-size:18px;">Administrator@yanzi /cygdrive/e/WorkSpaces/OpenCV4Android/FaceDetectLiu2 $ ndk-build Android NDK: WARNING: APP_PLATFORM android-19 is larger than android:minSdkVersion 16 in ./AndroidManifest.xml [armeabi-v7a] Cygwin : Generating dependency file converter script [armeabi-v7a] Compile++ thumb: ProcessImg <= DetectFace_JNI.cpp jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProcessImg_processIplImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:99:44: warning: converting 'false' to pointer type for argument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconversion-null] jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProcessImg_processStaticImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:133:44: warning: converting 'false' to pointer type for argument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconversion-null] [armeabi-v7a] Compile++ thumb: ProcessImg <= copyToAssets.cpp [armeabi-v7a] Compile++ thumb: ProcessImg <= detectFace.cpp [armeabi-v7a] SharedLibrary : libProcessImg.so [armeabi-v7a] Install : libProcessImg.so => libs/armeabi-v7a/libProcessImg.so </span></span>
上面两个警告么有关系,编译成功。生成的libProcessImg.so的大小为4M,整个apk大小为1.99M。
注意,如果将mk里的LOCAL_LDLIBS += -lm -llog这一句错误的写为:LOCAL_LDLIBS := -lm -llog,即将“+=”错写成了“:=”将会看到如下大量错误:
<span style="font-family: 'Comic Sans MS';"><span style="font-size:18px;">$ ndk-build Android NDK: WARNING: APP_PLATFORM android-19 is larger than android:minSdkVersion 16 in ./AndroidManifest.xml [armeabi-v7a] Compile++ thumb: ProcessImg <= DetectFace_JNI.cpp jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProcessImg_processIplImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:99:44: warning: converting 'false' to pointer type for argument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconversion-null] jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProcessImg_processStaticImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:133:44: warning: converting 'false' to pointer type for argument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconversion-null] [armeabi-v7a] Compile++ thumb: ProcessImg <= copyToAssets.cpp [armeabi-v7a] Compile++ thumb: ProcessImg <= detectFace.cpp [armeabi-v7a] SharedLibrary : libProcessImg.so D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function icvGets(CvFileStorage*, char*, int):persistence.cpp(.text._ZL7icvGetsP13CvFileStoragePci+0x7e): error: undefined reference to 'gzgets' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function icvXMLSkipSpaces(CvFileStorage*, char*, int):persistence.cpp(.text._ZL16icvXMLSkipSpacesP13CvFileStoragePci+0x1c4): error: undefined reference to 'gzeof' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function icvYMLSkipSpaces(CvFileStorage*, char*, int, int) [clone .constprop.65]:persistence.cpp(.text._ZL16icvYMLSkipSpacesP13CvFileStoragePcii.constprop.65+0x122): error: undefined reference to 'gzeof' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function icvPuts(CvFileStorage*, char const*):persistence.cpp(.text._ZL7icvPutsP13CvFileStoragePKc+0x32): error: undefined reference to 'gzputs' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function icvClose(CvFileStorage*, std::string*):persistence.cpp(.text._ZL8icvCloseP13CvFileStoragePSs+0x132): error: undefined reference to 'gzclose' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function cvOpenFileStorage:persistence.cpp(.text.cvOpenFileStorage+0x1ac): error: undefined reference to 'gzrewind' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function cvOpenFileStorage:persistence.cpp(.text.cvOpenFileStorage+0x6d4): error: undefined reference to 'gzclose' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function cvOpenFileStorage:persistence.cpp(.text.cvOpenFileStorage+0x75a): error: undefined reference to 'gzopen' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_core.a(persistence.cpp.o): in function cvOpenFileStorage:persistence.cpp(.text.cvOpenFileStorage+0xd80): error: undefined reference to 'gzclose' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../3rdparty/libs/armeabi-v7a/liblibpng.a(pngread.c.o): in function png_create_read_struct_2:pngread.c(.text.png_create_read_struct_2+0x112): error: undefined reference to 'inflateInit_' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../3rdparty/libs/armeabi-v7a/liblibpng.a(pngread.c.o): in function png_read_row:pngread.c(.text.png_read_row+0x218): error: undefined reference to 'inflate' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../3rdparty/libs/armeabi-v7a/liblibpng.a(pngread.c.o): in function png_read_destroy:pngread.c(.text.png_read_destroy+0x96): error: undefined reference to 'inflateEnd' D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../3rdparty/libs/armeabi-v7a/liblibpng.a(pngwrite.c.o): in function png_write_flush:pngwrite.c(.text.png_write_flush+0x1c): error: undefined reference to 'deflate' </span></span>
上两张运行效果图,分别是预览界面检测人脸和拍照后检测:
二、Android以JNI调OpenCV的第二种配置方法
Application.mk文件同上,Android.mk文件如下:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
OpenCV_INSTALL_MODULES:=on
OPENCV_CAMERA_MODULES:=off
OPENCV_LIB_TYPE:=SHARE
ifeq ("$(wildcard $(OPENCV_MK_PATH))","")
include D:\ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\OpenCV.mk
else
include $(OPENCV_MK_PATH)
endif
LOCAL_MODULE := ProcessImg
LOCAL_SRC_FILES := DetectFace_JNI.cpp \
src/copyToAssets.cpp \
src/detectFace.cpp
LOCAL_LDLIBS := -lm -llog
include $(BUILD_SHARED_LIBRARY)
唯一的变化时将OPENCV_LIB_TYPE:=STATIC 变成了SHARE.即通过动态链接的方式连接OpenCV的so。编译信息如下:
<span style="font-size:18px;">Administrator@yanzi /cygdrive/e/WorkSpaces/OpenCV4Android/FaceDetectLiu2 $ ndk-build Android NDK: WARNING: APP_PLATFORM android-19 is larger than android:minSdkVersi on 16 in ./AndroidManifest.xml Android NDK: WARNING:jni/Android.mk:ProcessImg: non-system libraries in linker f lags: -lopencv_java Android NDK: This is likely to result in incorrect builds. Try using LOCAL_S TATIC_LIBRARIES Android NDK: or LOCAL_SHARED_LIBRARIES instead to list the library dependenc ies of the Android NDK: current module [armeabi-v7a] Compile++ thumb: ProcessImg <= DetectFace_JNI.cpp jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProces sImg_processIplImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:99:44: warning: converting 'false' to pointer type for ar gument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconver sion-null] jni/DetectFace_JNI.cpp: In function '_jstring* Java_org_yan_processlib_LibProces sImg_processStaticImg(JNIEnv*, jobject, jintArray, int, int)': jni/DetectFace_JNI.cpp:133:44: warning: converting 'false' to pointer type for a rgument 2 of 'jint* _JNIEnv::GetIntArrayElements(jintArray, jboolean*)' [-Wconve rsion-null] [armeabi-v7a] Compile++ thumb: ProcessImg <= copyToAssets.cpp [armeabi-v7a] Compile++ thumb: ProcessImg <= detectFace.cpp [armeabi-v7a] SharedLibrary : libProcessImg.so D:/ProgramFile/android-ndk-r9d/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld.exe: warning: hidden symbol '__aeabi_atexit' in D:/ProgramFile/android-ndk-r9d/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi-v7a/libgnustl_static.a(atexit_arm.o) is referenced by DSO D:/ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\jni\/../libs/armeabi-v7a/libopencv_java.so [armeabi-v7a] Install : libProcessImg.so => libs/armeabi-v7a/libProcessImg.so </span>
可以看到上面说找不到non-system libraries in linker flags: -lopencv_java这个东西,关于这个问题我曾作如下尝试:
LOCAL_LDLIBS += -lopencv_java 或 LOCAL_SHARED_LIBRARIES += libopencv_java均没有解决这个warning。原本运行正常的程序报错如下:
java.lang.UnsatisfiedLinkError: Cannot load library: soinfo_link_image(linker.cpp:1635): could not load library "libopencv_java.so" needed by "libProcessImg.so"; caused by load_library(linker.cpp:745): library "libopencv_java.so" not found
说是自己编译的这个库libProcessImg.so依赖于libopencv_java.so,没有找到它所以程序挂了。再看生成的libProcessImg.so大小为437KB,比第一种方式少了好几倍啊。肿么让程序正常运行呢?将安装目录D:\ProgramFile\OpenCV-2.4.9-android-sdk\sdk\native\libs\armeabi-v7a下的libopencv_java.so拷贝到libs\armeabi-v7a文件夹下,然后再调用库的时候方法变更为:
<span style="font-size:18px;">package org.yan.processlib; public class LibProcessImg { static{ System.loadLibrary("opencv_java"); System.loadLibrary("ProcessImg"); } public static native void initProcessLib(String str); public static native String processIplImg(int[] buf, int w, int h); public static native String processStaticImg(int[] buf, int w, int h); } </span>
先调用这个依赖的库,然后调用我们自己的,注意这个libopencv_java.so有9M多。如此程序又可以正常运行了。最终apk的大小为4.83M,是第一种的2倍,但还没大的离谱,可以接受。
对比上面两种方法不难发现,虽然在mk里都有include $(BUILD_SHARED_LIBRARY) 也即生成的库都是动态库,但这个库指的是我们自己写的,我们的库要进一步调用OpenCV的库,否则的话直接就能用OpenCV库里的函数,咋可能有这事呢。至于怎么调OpenCV的库,可以静态,也可以动态。这也就是为什么第二种方法生成的so的大小只有437KB,而第一种方法生成的库有4M的大小。事实上在我们第一种方式ndk-build的时候会发现有大量的各种.a .a被加载进去的情形,只不过这只出现一次,原因就在这。打开安装目录下的libs:
可以看到除了带Camera的so外,其他大量都是.a,而且这些.a是按包名划分的。而so只有libopencv_java.so和libopencv_info.so,在功能上这些.a静态调用时等同于动态加载这两个so。之所以这么多.a就是供我们采用第一种方法时使用的。关于静态和动态的优缺点参见这里
第三种方法:java和jni混用
搞完第二种,既然动态加载我表面上没用libopencv_java.so,还要把它加载进来,那我干脆为啥不用用java的api呢?既然要用java的api那肯定要jar包弄进来,于是导进来OpenCV Library - 2.4.9工程如下图所示:
这样就可以用java的api了。如果你有强迫症,觉得OpenCV Library - 2.4.9这个工程一直开着心里不爽,那也可以将sdk bin目录生成的opencv library - 2.4.9.jar拷贝到自己工程的libs文件夹下,记得将刚才添加的Libraries remove掉。右键opencv library - 2.4.9.jar----------build path--------------add to build path,这样照样使用Java的api。其实这块很明显,只要jar包弄进去了你就可以正常使用api了,即编译时不报错,但apk到手机上能不能正常运行则是另一码子事。
此时的mk文件跟第二种类似,记得把libopencv_java.so拷贝到相应目录。相较于第二种,并没有增加什么,仅仅是开发时将jar包导入就可以正常编译了,能否正常运行还依赖于libopencv_java.so。需注意的是,每clean一次,这个libopencv_java.so就会不见一次,还要手动拷或者自己写个脚本拷。最终apk的大小为4.94M,相比第二种多点,原因是那个jar包的原因,以及我们代码里又加了几句:
<span style="font-size:18px;">package org.yan.processlib; import org.opencv.android.OpenCVLoader; import android.util.Log; public class LibProcessImg { static{ if(!OpenCVLoader.initDebug()){ Log.i("yanzi", "OpenCVLoader.initDebug() 失败"); }else{ System.loadLibrary("opencv_java"); System.loadLibrary("ProcessImg"); } } public static native void initProcessLib(String str); public static native String processIplImg(int[] buf, int w, int h); public static native String processStaticImg(int[] buf, int w, int h); } </span>
当然你可以加其他的很多OpenCV的java接口,比如Bitmap转mat,直接传Mat指针到jni等等,随便自己怎么玩。官网上的JNI使用OpenCV其实就是这种java和jni混用的情况,其实大多情况下么有啥必要,看个人了。至于动态加载OpenCV的库还是静态,也全看个人,我是倾向于第一种,以apk的体积最清爽为准。
我们用initDebug一下,其实这块你不写也行的。另外就是这个加载库用static方法跟放Activity里的onResume里差不多,我是习惯了放单独的一个静态方法里。记住千万不要用OpenCVLoader.initAsync()方法啊,本文的主线就是不用OpenCVManager!!!
最后我们打开一个OpenCV_2.4.9_Manager_2.18_armv7a-neon.apk来看一下:
哈哈,看到了吧,里面的精髓就是lib下的so以及那个引擎so。在使用OpenCVManager的情况下,这些库随着OpenCVManager.apk的装入都事先安装到手机了,不论是使用java也好,还是用jni再使用动态链接OpenCV库的方法(使自己的so体积最小),都不用往libs文件夹额外加so了,因为so随着OpenCVManager已安好了。这就是之所以加个OpenCVManager的半个初衷啊,另半个初衷是binder service 框架上的原因!!!
最后补充3点:
1.有时Cygwin会有记忆效应,比如你修改了mk里从static变成share,但是它还是按照static来编译的。解决方法是退了重新进,或重启电脑吧,汗。
2.除了ndk-build命令外,还应该记住ndk-build -B 强制全编 和 ndk-build clean 清理 这两个命令。
3.有些教程用到jni时还要把工程转成C++工程,再配置ndk-build.cmd命令,其实这个在前文也曾说过。个人觉得真心么必要啊。
------------------------本文系原创,转载请注明作者:yanzi1225627