Android开发笔记[18]-使用本地模块
摘要
将摄像头预览页面封装到Android模块中并在app中使用这个本地模块.
关键信息
- Android Studio:Iguana | 2023.2.1
- Gradle:distributionUrl=https://services.gradle.org/distributions/gradle-8.4-bin.zip
- jvmTarget = '1.8'
- minSdk 26
- targetSdk 34
- compileSdk 34
- 开发语言:Kotlin,Java
- ndkVersion = '21.1.6352462'
- kotlin版本:1.9.20
- kotlinCompilerExtensionVersion '1.5.4'
- com.android.library:8.3
原理简介
CameraX简介
[https://juejin.cn/post/6895278749630070798]
CameraX 是一个 Jetpack 支持库，旨在帮助您简化相机应用的开发工作。
它提供一致且易于使用的 API Surface，适用于大多数 Android 设备，并可向后兼容至 Android 5.0
基于Camera2，即对Camera2的封装.
Android模块开发简介
[https://www.jianshu.com/p/0ea37b2c7ce7]
模块化开发思路就是：单独开发每个模块，用集成的方式把他们组合起来，就能拼出一个app。app可以理解成很多功能模块的组合，而且有些功能模块是通用的，必备的，像自动更新，反馈，推送，都可以提炼成模块，和搭积木很像，由一个壳包含很多个模块。
Android使用OpenCV库的简单方式
[https://github.com/QuickBirdEng/opencv-android]
Easy way to integrate OpenCV into your Android project via Gradle.
No NDK dependency needed - just include this library and you are good to go.
OpenCV Contribution's package naming has been changed to make it as per the naming guideline.
Old: opencv:VERSION-contrib
New: opencv-contrib:VERSION
Each versions is available in only OpenCV as well as OpenCV with contributions.
‼️ Please use 4.5.3.0 instead of 4.5.3. They are both the same versions, however, 4.5.3 has some runtime issues on some of the Android versions while 4.5.3.0 works fine.
实现
核心代码
- Android Studio 文件->New->New Module新建安卓模块
grape_realtime_detect - 编辑模块的build.gradle.kts文件
repositories {
mavenCentral()
}
implementation `com.quickbirdstudios:opencv:4.5.3.0`
- app添加模块:
settings.gradle
include ':app:grape_realtime_detect'
build.gradle
implementation ('com.github.zynkware:Document-Scanning-Android-SDK:1.1.1') /* 文档扫描库*/ {
// 去除OpenCV冲突依赖
exclude group: "com.github.zynkware", module: "Tiny-OpenCV"
}
/* start 实时识别相关 */
implementation project(':app:grape_realtime_detect')
/* end 实时识别相关 */
- 编辑模块的AndroidManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-feature android:name="android.hardware.camera2" android:required="false"/>
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<application>
<!-- 目标检测页模板 -->
<activity
android:name=".RealtimeDetectActivityTemplate"
android:exported="true"
android:label="@string/title_activity_realtime_detect"
android:theme="@style/Theme.Grapeyolov5detectandroid" />
</application>
</manifest>
- 模块代码
RealtimeDetectActivityTemplate.kt
package cn.qsbye.grape_realtime_detect
import android.app.Activity
import android.content.Context
import android.content.res.AssetManager
import android.graphics.BitmapFactory
import android.os.Bundle
import android.util.Log
import android.view.ViewGroup
import android.widget.Toast
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.annotation.OptIn
import androidx.camera.core.CameraSelector
import androidx.camera.core.ExperimentalGetImage
import androidx.camera.core.ImageCapture
import androidx.camera.core.ImageCapture.CAPTURE_MODE_MAXIMIZE_QUALITY
import androidx.camera.core.ImageCaptureException
import androidx.camera.core.ImageProxy
import androidx.camera.core.UseCase
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.video.Recorder
import androidx.camera.view.PreviewView
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.MutatePriority
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxHeight
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.width
import androidx.compose.material3.Button
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.MutableState
import androidx.compose.runtime.compositionLocalOf
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateListOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.ImageBitmap
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.graphics.nativeCanvas
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalLifecycleOwner
import androidx.compose.ui.text.drawText
import androidx.compose.ui.text.font.Font
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.camera.core.Preview as PreviewCameraX
import androidx.compose.ui.viewinterop.AndroidView
import androidx.core.content.ContextCompat
import cn.qsbye.grape_realtime_detect.ui.theme.Grapeyolov5detectandroidTheme
import com.hjq.permissions.OnPermissionCallback
import com.hjq.permissions.Permission
import com.hjq.permissions.XXPermissions
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import org.opencv.videoio.VideoCapture
import java.util.concurrent.Executor
import kotlin.coroutines.resume
import kotlin.coroutines.suspendCoroutine
import org.opencv.core.Core
import org.opencv.core.Mat
import org.opencv.core.Scalar
import org.opencv.imgcodecs.Imgcodecs
import java.io.File
// 全局引用Activity
val LocalActivity = compositionLocalOf<Activity?> { null }
/* start 扩展Context */
// 添加相机画面
suspend fun Context.getCameraProvider(): ProcessCameraProvider = suspendCoroutine { continuation ->
ProcessCameraProvider.getInstance(this).also { future ->
future.addListener({
continuation.resume(future.get())
}, executor)
}
}
val Context.executor: Executor
get() = ContextCompat.getMainExecutor(this)
/* end 扩展Context */
abstract class RealtimeDetectActivityTemplate : ComponentActivity() {
// 显示Toast消息
private fun Context.toast(message: CharSequence) =
Toast.makeText(this, message, Toast.LENGTH_SHORT).show();
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
/* start 检查相机权限 */
XXPermissions.with(this)
// 申请单个权限
.permission(Permission.CAMERA)
.request(object : OnPermissionCallback {
override fun onGranted(permissions: MutableList<String>, allGranted: Boolean) {
if (!allGranted) {
// toast("相机权限正常")
return
}
toast("获取相机权限成功")
}
override fun onDenied(permissions: MutableList<String>, doNotAskAgain: Boolean) {
if (doNotAskAgain) {
toast("被永久拒绝授权,请手动授予相机权限")
// 如果是被永久拒绝就跳转到应用权限系统设置页面
XXPermissions.startPermissionActivity(this@RealtimeDetectActivityTemplate, permissions)
} else {
toast("获取相机权限失败")
// 结束当前活动
finish()
}
}
})
/* end 检查相机权限 */
// 初始化assets文件夹实例
val assetManager: AssetManager = assets
setContent {
Grapeyolov5detectandroidTheme {
// A surface container using the 'background' color from the theme
Surface(
modifier = Modifier.fillMaxSize(),
color = MaterialTheme.colorScheme.background
) {
provideActivity(activity = this) {
realtimeDetectPage()
}
}
}
}// end setContent
}
}
// 实时检测页面布局
@Preview(showBackground = true)
@Composable
fun realtimeDetectPage(_modifier: Modifier = Modifier){
Column (
_modifier.fillMaxSize(),
verticalArrangement = Arrangement.Bottom,
// horizontalAlignment = Alignment.CenterHorizontally
){
cameraArea(_modifier = Modifier.weight(0.8f))
navigationBarArea(
_modifier = Modifier
.weight(1f)
)
}
}
// 实时显示区域
@Composable
fun cameraArea(_modifier: Modifier = Modifier, onUseCase: (UseCase) -> Unit = { }, onImageFileSaveSucceed:(File)-> Unit={
}){
// 获取上下文
val context = LocalContext.current
// 协程视图
val coroutineScope = rememberCoroutineScope()
// 绑定当前生命周期
val lifecycleOwner = LocalLifecycleOwner.current
// 相机缩放方式
val scaleType: PreviewView.ScaleType = PreviewView.ScaleType.FILL_CENTER
// 选定使用后置摄像头
val cameraSelector: CameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
// 图片预览实例
var previewUseCase by remember { mutableStateOf<UseCase>(PreviewCameraX.Builder().build()) }
// 图片捕捉实例
val imageCaptureUseCase by remember {
mutableStateOf(
ImageCapture.Builder()
.setCaptureMode(CAPTURE_MODE_MAXIMIZE_QUALITY)
.build()
)
}
Box(
modifier = _modifier
){
// 图像Bitmap状态
var imageBitmap by remember { mutableStateOf<ImageBitmap?>(null) }
// 使用remember来记忆boxes数组和isRandom变量的变化
val s_boxes = remember { mutableStateOf(arrayOf(arrayOf<Float>())) }
val s_isRandom = remember { mutableStateOf<Boolean>(false) }
// 获取上下文
val context2 = LocalContext.current
/* start 摄像头画面Preview */
AndroidView(
modifier = _modifier,
factory = { context ->
val previewView = PreviewView(context).apply {
this.scaleType = scaleType
layoutParams = ViewGroup.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.MATCH_PARENT
)
}
// CameraX Preview UseCase
// 这里注意导入androidx.camera.core.Preview as PreviewCameraX而不是androidx.compose.ui.tooling.preview.Preview
previewUseCase = PreviewCameraX.Builder()
.build()
.also {it->
it.setSurfaceProvider(previewView.surfaceProvider)
}
// 协程启动
coroutineScope.launch {
val cameraProvider = context.getCameraProvider()
try {
// 在下一次绑定之前需要解绑上一个
cameraProvider.unbindAll()
cameraProvider.bindToLifecycle(
lifecycleOwner, cameraSelector, previewUseCase, imageCaptureUseCase
)
} catch (ex: Exception) {
Log.e("CameraPreview", "Use case binding failed", ex)
}
}
// 返回previewView画面
previewView
}
)
/* end 摄像头画面Preview */
/* start 预览图片 */
// 绘制图像
imageBitmap?.let { bitmap ->
Canvas(modifier = _modifier
.height(120.dp)
.width(90.dp)) {
drawImage(
image = bitmap,
topLeft = Offset(x = 0.dp.toPx(), y = 0.dp.toPx()),
alpha = 0.5f, // 设置透明度为50%
)
}
}
/* end 预览图片 */
/* start 绘制矩形框 */
// 使用remember监听boxes变量和isRandom变量变化并更新显示
s_boxes.value = arrayOf(
arrayOf(100f, 200f, 400f, 500f),
arrayOf(50f, 150f, 300f, 450f),
arrayOf(20f, 300f, 100f, 500f)
)
drawRectangleBox(_modifier = _modifier, isRandom = s_isRandom.value, boxes = s_boxes.value)
/* end 绘制矩形框 */
/* start 拍照协程 */
LaunchedEffect(Unit) {
coroutineScope.launch(Dispatchers.IO) {
while (true) {
try {
/* start 调用takePicture */
imageCaptureUseCase.takePicture(
context.executor, // 执行器
object : ImageCapture.OnImageCapturedCallback() {
fun calculateInSampleSize(options: BitmapFactory.Options, reqWidth: Int, reqHeight: Int): Int {
// 计算实际尺寸
val (actualWidth, actualHeight) = options.run { outWidth to outHeight }
// 计算缩放因子
var inSampleSize = 1
if (actualWidth > reqWidth || actualHeight > reqHeight) {
val halfWidth = actualWidth / 2
val halfHeight = actualHeight / 2
inSampleSize = if (halfWidth < reqWidth || halfHeight < reqHeight) {
halfWidth.coerceAtLeast(halfHeight).toInt()
} else {
halfWidth.coerceAtMost(halfHeight).toInt()
}
}
// 计算最终的缩放因子
inSampleSize = Math.round(Math.pow(2.0, ((Integer.SIZE - Integer.numberOfLeadingZeros(
(inSampleSize - 1)
)).toDouble()))).toInt()
return inSampleSize
}
@OptIn(ExperimentalGetImage::class)
override fun onCaptureSuccess(imageProxy: ImageProxy) {
// 图片捕获成功，可以在这里处理ImageProxy
Log.d("CameraX", "图片捕获成功")
// 将ImageProxy转换为Bitmap
val image = imageProxy.image
val buffer = image!!.planes[0].buffer
val bytes = ByteArray(buffer.remaining())
buffer.get(bytes)
val options = BitmapFactory.Options()
options.inJustDecodeBounds = true
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, options)
options.inSampleSize = calculateInSampleSize(options, 256, 256)
options.inJustDecodeBounds = false
val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size, options)
//val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
// 将Bitmap转换为ImageBitmap
val _imageBitmap = bitmap?.asImageBitmap()
// 更新图像状态
imageBitmap = _imageBitmap
// 调用TFLite识别葡萄
LocalActivity.run{
s_boxes.value=GrapeDetect.detectGrapes(imageBitmap!!,context2)
}
Log.d("GrapeRealtimeDetect", "识别葡萄完成!")
// 释放资源
imageProxy.close()
} // end imageProxy
override fun onError(exception: ImageCaptureException) {
// 处理捕获过程中的错误
Log.d("CameraX", "图片捕获失败: ${exception.message}")
}
}
)
/* end 调用takePicture */
} catch (e: Exception) {
// 处理异常情况
Log.d("CameraX","图片捕获异常")
}
// 等待500毫秒
delay(500)
}
}
}
/* end 拍照协程 */
} // end Box
}
// 返回按钮区域
@Composable
fun navigationBarArea(_modifier: Modifier = Modifier){
// 获取当前上下文
val context = LocalContext.current
// 获取当前活动
val activity = LocalActivity.current
Column(
modifier = Modifier
.background(Color(0xFFDBDF74))
.fillMaxWidth(),
horizontalAlignment= Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center
){
Button(
modifier = Modifier
.fillMaxWidth(0.8f),
onClick = {
// 关闭当前Activity
activity?.finish()
}){
Text(
text = "返回"
)
}
}
}
// 提供当前活动
@Composable
fun provideActivity(activity: Activity, content: @Composable () -> Unit) {
CompositionLocalProvider(LocalActivity provides activity) {
content()
}
}
// 绘制矩形框
@Composable
fun drawRectangleBox(_modifier : Modifier,isRandom: Boolean = false, boxes: Array<Array<Float>>) {
Canvas(modifier = _modifier.fillMaxSize()) {
// 遍历每个矩形框
for ((startX, startY, endX, endY) in boxes) {
// 判断isRandom，选择颜色
var color = if (isRandom) Color.Red else Color.Green
// 绘制空心矩形框
drawRect(
color = color,
// start = Offset(startX, startY),
size = Size(endX - startX, endY - startY),
style = Stroke(width = 5f) // 设置边框宽度
)
// 计算矩形框的中心点
val centerX = startX + (endX - startX) / 2
val centerY = startY + (endY - startY) / 2
class TextMeasurer {
fun measureText(text: String, fontSize: Float, font: Font): Size {
// 这里是测量文本尺寸的代码
// 可能需要根据实际的图形库或API来编写
// 返回文本的宽度和高度
return Size(50f, 30f)
}
}
// 使用 TextMeasurer 来测量文本尺寸
val textMeasurer = TextMeasurer()
val nativePaint = android.graphics.Paint().let {
it.apply {
textSize = 36f
color = Color(0,255,0)
}
}
// 绘制文字
drawContext.canvas.nativeCanvas.drawText(
"葡萄",
centerX,
centerY,
nativePaint
)
}
}
}
GrapeRealtimeDetect.kt
package cn.qsbye.grape_realtime_detect
import android.content.Context
import android.content.res.AssetManager
import android.graphics.Bitmap
import android.graphics.RectF
import android.util.Log
import androidx.compose.ui.graphics.ImageBitmap
import org.opencv.android.OpenCVLoader
import kotlin.random.Random
/*
GrapeRealtimeDetect模块入口
*/
object GrapeRealtimeDetect {
// 初始化模块
fun init() {
/* start 初始化OpenCV库 */
Log.d("GrapeRealtimeDetect","开始初始化!")
if (!OpenCVLoader.initDebug()) {
Log.d("GrapeRealtimeDetect", "无法加载OpenCV库!");
} else {
Log.d("GrapeRealtimeDetect", "OpenCV加载成功!");
}
/* end 初始化OpenCV库 */
}
}
/* start 葡萄检测 */
// 葡萄识别总函数
object GrapeDetect {
fun detectGrapes(imageBitmap: ImageBitmap,context: Context): Array<Array<Float>> {
// TODO 调用TFLite识别葡萄
// 假设这里会有一些图像处理的代码来识别葡萄
// ...
// 随机生成矩形的宽度和高度（>=100f）
val width = Random.nextFloat() * (imageBitmap.width - 100) + 100
val height = Random.nextFloat() * (imageBitmap.height - 100) + 100
// 随机生成矩形的坐标
val x = Random.nextFloat() * (imageBitmap.width - width)
val y = Random.nextFloat() * (imageBitmap.height - height)
// 更新 s_boxes 状态
return arrayOf(arrayOf(x, y, x + width, y + height))
}
}
// 图像预处理函数
private fun preprocessImage(bitmap: Bitmap): Bitmap {
// 这里应根据模型的要求对图像进行预处理
// 例如，调整图像大小、归一化等
return bitmap
}
// TensorFlow Lite模型加载和推理的辅助类
class TfLiteModelLoader {
companion object {
// fun loadModelFromFile(context: Context, modelName: String): TfLiteModel {
// // 实现模型加载逻辑
//
// return null
// }
}
}
class TfLiteModel {
// fun recognizeImage(image: Bitmap): List<RecognitionResult> {
// // 实现推理逻辑，并返回检测结果
// return null
// }
}
// 检测结果的数据类
data class RecognitionResult(
val boundingBox: RectF,
// 其他可能的字段，如类别、置信度等
)
/* end 葡萄检测 */
build.gradle.kts
import com.android.build.api.dsl.AaptOptions
plugins {
id("com.android.library")
id("org.jetbrains.kotlin.android")
}
android {
namespace = "cn.qsbye.grape_realtime_detect"
compileSdk = 34
defaultConfig {
minSdk = 24
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
consumerProguardFiles("consumer-rules.pro")
vectorDrawables {
useSupportLibrary = true
}
}
buildTypes {
release {
isMinifyEnabled = false
proguardFiles(
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}
kotlinOptions {
jvmTarget = "1.8"
}
buildFeatures {
compose = true
}
composeOptions {
kotlinCompilerExtensionVersion = "1.5.4"
}
packagingOptions {
resources {
excludes += "/META-INF/{AL2.0,LGPL2.1}"
}
}
aaptOptions {
// 不压缩模型文件
noCompress("tflite")
noCompress("lite")
}
}
dependencies {
/* start Android系统相关 */
implementation("androidx.core:core-ktx:1.13.0")
implementation("androidx.appcompat:appcompat:1.6.1")
implementation("com.google.android.material:material:1.11.0")
implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.7.0")
implementation("androidx.activity:activity-compose:1.8.2")
implementation(platform("androidx.compose:compose-bom:2024.04.00"))
implementation("androidx.compose.ui:ui")
implementation("androidx.compose.ui:ui-graphics")
implementation("androidx.compose.ui:ui-tooling-preview")
implementation("androidx.compose.material3:material3")
testImplementation("junit:junit:4.13.2")
androidTestImplementation("androidx.test.ext:junit:1.1.5")
androidTestImplementation("androidx.test.espresso:espresso-core:3.5.1")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")
/* end Android系统相关 */
/* start 相机相关 */
implementation("io.reactivex.rxjava3:rxandroid:3.0.0")
implementation("androidx.camera:camera-camera2:1.3.3")
implementation("androidx.camera:camera-lifecycle:1.3.3")
implementation("androidx.camera:camera-view:1.3.3")
implementation("androidx.camera:camera-core:1.3.3")
implementation("com.google.android.exoplayer:exoplayer-core:2.19.1")
/* end 相机相关 */
/* start 图像处理相关 */
//implementation("com.github.zynkware:Tiny-OpenCV:4.4.0-4")
implementation("com.quickbirdstudios:opencv-contrib:4.5.3.0")
/* end 图像处理相关 */
/* start 数据同步框架相关 */
implementation("com.tencent:mmkv:1.3.4")
/* end 数据同步框架相关 */
/* start 权限相关 */
implementation("com.github.getActivity:XXPermissions:18.6")
androidTestImplementation(platform("androidx.compose:compose-bom:2024.04.00"))
androidTestImplementation("androidx.compose.ui:ui-test-junit4")
debugImplementation("androidx.compose.ui:ui-tooling")
debugImplementation("androidx.compose.ui:ui-test-manifest") // 动态权限库
/* end 权限相关 */
/* start 测试相关 */
testImplementation("androidx.test.ext:junit:1.1.5")
testImplementation("androidx.test:rules:1.5.0")
testImplementation("androidx.test:runner:1.5.2")
testImplementation("androidx.test.espresso:espresso-core:3.5.1")
testImplementation("org.robolectric:robolectric:4.4")
/* end 测试相关 */
/* start tflite相关 */
implementation("androidx.window:window:1.2.0")
implementation("org.tensorflow:tensorflow-lite-task-vision:0.4.0")
implementation("org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.0")
implementation("org.tensorflow:tensorflow-lite-gpu:2.9.0")
/* end tflite相关 */
}
- 编辑app的初始化代码以初始化模块
BootApp.kt
class BootApp:Application() {
companion object {
private const val FILE_SIZE = 1000000L
private const val FILE_QUALITY = 100
private val FILE_TYPE = Bitmap.CompressFormat.JPEG
}
override fun onCreate() {
super.onCreate()
/* start 初始化实时识别模块 */
GrapeRealtimeDetect.init()
Log.d("GrapeRealtimeDetect","初始化完成!")
/* end 初始化实时识别模块 */
}
}
效果
| 模块中调用CameraX相机 |
|---|
 |
