[Artoolkit] Android Sample of nftSimple
结合：[Artoolkit] ARToolKit's SDK Structure on Android
重难点：aRBaseLib/, nftSimple/, libcpufeatures.a
- aRBaseLib/
如果摄像头控制只能在android层操作，那么ARBaseLib中的摄像头控制部分就是重点之一。
unsw@unsw-UX303UB$ _cmd-wc-java 204 ./AndroidUtils.java 395 ./NativeInterface.java 429 ./ARToolKit.java 242 ./camera/CaptureCameraPreview.java 420 ./camera/CameraPreferencesActivity.java 211 ./camera/CameraWrapper.java 70 ./camera/CameraEventListener.java 106 ./rendering/gles20/ARRendererGLES20.java 85 ./rendering/gles20/LineGLES20.java 89 ./rendering/gles20/BaseFragmentShader.java 196 ./rendering/gles20/ShaderProgram.java 83 ./rendering/gles20/BaseShaderProgram.java 84 ./rendering/gles20/CubeGLES20.java 60 ./rendering/gles20/OpenGLShader.java 53 ./rendering/gles20/ARDrawableOpenGLES20.java 102 ./rendering/gles20/BaseVertexShader.java 139 ./rendering/Cube.java 93 ./rendering/ARRenderer.java 83 ./rendering/RenderUtils.java 152 ./rendering/Line.java 51 ./assets/AssetFileTransferException.java 52 ./assets/HashComputationException.java 191 ./assets/AssetFileTransfer.java 154 ./assets/Hasher.java 222 ./assets/AssetHelper.java 122 ./FPSCounter.java 433 ./ARActivity.java 4521 total unsw@unsw-UX303UB$
4000行代码，从摄像头控制开始。(Replaced with opencv4android)
01. rendering.ARActivity是main()，其中：
CameraEventListener接口可替换为opencv4android提供的CameraBridgeViewBase.CvCameraViewListener2接口。
02. 处理每一帧
@Override public void cameraPreviewFrame(byte[] frame) { if (firstUpdate) { // ARToolKit has been initialised. The renderer can now add markers, etc... if (renderer.configureARScene()) { Log.i(TAG, "Scene configured successfully"); } else { // Error Log.e(TAG, "Error configuring scene. Cannot continue."); finish(); } firstUpdate = false; } if (ARToolKit.getInstance().convertAndDetect(frame)) { // Update the renderer as the frame has changed if (glView != null) glView.requestRender(); onFrameProcessed(); } }
03. ARToolKit处理帧，然后调用native（NDK）
public boolean convertAndDetect(byte[] frame) { if (!initedNative) return false; if (frame == null) return false; if (!NativeInterface.arwAcceptVideoImage(frame, frameWidth, frameHeight, cameraIndex, cameraIsFrontFacing)) return false; if (!NativeInterface.arwCapture()) return false; return NativeInterface.arwUpdateAR(); }
libARWrapper.so的native代码在哪里？
nftSimple没有使用Wrapper技术，而是直接调用native: "AR and Rendering Code in Native C/C++ Using Android NDK"
怎么感觉 aRBaseLib 没有大用 ?
@Override public void onPreviewFrame(byte[] data, Camera cam) { nftSimpleActivity.nativeVideoFrame(data); cam.addCallbackBuffer(data); }
04. 一帧的处理
1 JNIEXPORT void JNICALL JNIFUNCTION_NATIVE(nativeVideoFrame(JNIEnv* env, jobject obj, jbyteArray pinArray)) 2 { 3 int i, j, k; 4 jbyte* inArray; 5 6 if (!videoInited) { 7 #ifdef DEBUG 8 LOGD("nativeVideoFrame !VIDEO\n"); 9 #endif 10 return; // No point in trying to track until video is inited. 11 } 12 if (!nftDataLoaded) { 13 if (!nftDataLoadingThreadHandle || threadGetStatus(nftDataLoadingThreadHandle) < 1) { 14 #ifdef DEBUG 15 LOGD("nativeVideoFrame !NFTDATA\n"); 16 #endif 17 return; 18 } else { 19 nftDataLoaded = true; 20 threadWaitQuit(nftDataLoadingThreadHandle); 21 threadFree(&nftDataLoadingThreadHandle); // Clean up. 22 } 23 } 24 if (!gARViewInited) { 25 return; // Also, we won't track until the ARView has been inited. 26 #ifdef DEBUG 27 LOGD("nativeVideoFrame !ARVIEW\n"); 28 #endif 29 } 30 #ifdef DEBUG 31 LOGD("nativeVideoFrame\n"); 32 #endif 33 34 // Copy the incoming YUV420 image in pinArray. 35 env->GetByteArrayRegion(pinArray, 0, gVideoFrameSize, (jbyte *)gVideoFrame); 36 37 // As of ARToolKit v5.0, NV21 format video frames are handled natively, 38 // and no longer require colour conversion to RGBA. 39 // If you still require RGBA format information from the video, 40 // here is where you'd do the conversion: 41 // color_convert_common(gVideoFrame, gVideoFrame + videoWidth*videoHeight, videoWidth, videoHeight, myRGBABuffer); 42 43 videoFrameNeedsPixelBufferDataUpload = true; // Note that buffer needs uploading. (Upload must be done on OpenGL context's thread.) 44 45 // Run marker detection on frame 46 if (trackingThreadHandle) { 47 // Perform NFT tracking. 48 float err; 49 int ret; 50 int pageNo; 51 // 又到了熟悉的地方：） 52 if( detectedPage == -2 ) { 53 trackingInitStart( trackingThreadHandle, gVideoFrame ); 54 detectedPage = -1; 55 } 56 if( detectedPage == -1 ) { 57 ret = trackingInitGetResult( trackingThreadHandle, trackingTrans, &pageNo); 58 if( ret == 1 ) { 59 if (pageNo >= 0 && pageNo < surfaceSetCount) { 60 #ifdef DEBUG 61 LOGE("Detected page %d.\n", pageNo); 62 #endif 63 detectedPage = pageNo; 64 ar2SetInitTrans(surfaceSet[detectedPage], trackingTrans); 65 } else { 66 LOGE("Detected bad page %d.\n", pageNo); 67 detectedPage = -2; 68 } 69 } else if( ret < 0 ) { 70 #ifdef DEBUG 71 LOGE("No page detected.\n"); 72 #endif 73 detectedPage = -2; 74 } 75 } 76 if( detectedPage >= 0 && detectedPage < surfaceSetCount) { 77 if( ar2Tracking(ar2Handle, surfaceSet[detectedPage], gVideoFrame, trackingTrans, &err) < 0 ) { 78 #ifdef DEBUG 79 LOGE("Tracking lost.\n"); 80 #endif 81 detectedPage = -2; 82 } else { 83 #ifdef DEBUG 84 LOGE("Tracked page %d (max %d).\n", detectedPage, surfaceSetCount - 1); 85 #endif 86 } 87 } 88 } else { 89 LOGE("Error: trackingThreadHandle\n"); 90 detectedPage = -2; 91 } 92 93 // Update markers. 94 for (i = 0; i < markersNFTCount; i++) { 95 markersNFT[i].validPrev = markersNFT[i].valid; 96 if (markersNFT[i].pageNo >= 0 && markersNFT[i].pageNo == detectedPage) { 97 markersNFT[i].valid = TRUE; 98 for (j = 0; j < 3; j++) for (k = 0; k < 4; k++) markersNFT[i].trans[j][k] = trackingTrans[j][k]; 99 } 100 else markersNFT[i].valid = FALSE; 101 if (markersNFT[i].valid) { 102 103 // Filter the pose estimate. 104 if (markersNFT[i].ftmi) { 105 if (arFilterTransMat(markersNFT[i].ftmi, markersNFT[i].trans, !markersNFT[i].validPrev) < 0) { 106 LOGE("arFilterTransMat error with marker %d.\n", i); 107 } 108 } 109 110 if (!markersNFT[i].validPrev) { 111 // Marker has become visible, tell any dependent objects. 112 //ARMarkerAppearedNotification 113 } 114 115 // We have a new pose, so set that. 116 arglCameraViewRHf(markersNFT[i].trans, markersNFT[i].pose.T, 1.0f /*VIEW_SCALEFACTOR*/); 117 // Tell any dependent objects about the update. 118 //ARMarkerUpdatedPoseNotification 119 120 } else { 121 122 if (markersNFT[i].validPrev) { 123 // Marker has ceased to be visible, tell any dependent objects. 124 //ARMarkerDisappearedNotification 125 } 126 } 127 } 128 }
由原来的 mainloop 变为了android preview 一帧来调用，其他native部分都相同。
- libcpufeatures.a
Link: https://developer.android.com/ndk/guides/cpu-features.html
Android 官方提供的读取cpu信息的一个库。我们这里主要对应于线程相关的部分。
Android Studio 2.2.3直接加载Android example出现的问题：
|
@PN1019, this is a tough nut. Let me try summarize and hopefully that'll help you. So there's two ways to get around this. Use the JDK that is embedded with Android Studio 2.2.+. Or, tell Android Studio to use the Oracle Corp JDK installed on your devbox but have it run in such a way to regress in compiling an older version of the Java language creating an older version of runtime Java classes. I recommend the former, please see my entry above now labeled with "Recommended JDK to use:". |
and Android Studio 2.1.1 is better.
unsw@unsw-UX303UB$ sudo update-alternatives --config javac There are 3 choices for the alternative javac (providing /usr/bin/javac). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/jvm/java-7-oracle/bin/javac 1082 auto mode * 1 /usr/lib/jvm/java-7-oracle/bin/javac 1082 manual mode 2 /usr/lib/jvm/java-8-openjdk-amd64/bin/javac 1081 manual mode 3 /usr/lib/jvm/java-8-oracle/bin/javac 1081 manual mode Press enter to keep the current choice[*], or type selection number: 1 unsw@unsw-UX303UB$ sudo update-alternatives --config java There are 4 choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/jvm/java-7-oracle/jre/bin/java 1082 auto mode * 1 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java 1071 manual mode 2 /usr/lib/jvm/java-7-oracle/jre/bin/java 1082 manual mode 3 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode 4 /usr/lib/jvm/java-8-oracle/jre/bin/java 1081 manual mode Press enter to keep the current choice[*], or type selection number: 1 unsw@unsw-UX303UB$ javac -version javac 1.7.0_80 unsw@unsw-UX303UB$ java -version java version "1.7.0_121" OpenJDK Runtime Environment (IcedTea 2.6.8) (7u121-2.6.8-1ubuntu0.14.04.3) OpenJDK 64-Bit Server VM (build 24.121-b00, mixed mode)
Next: how to implement multi-nft，start with linux verison.
