使用JNI在Android图像解码和处理图像、JNI、Android

2023-09-05 10:52:55 作者:萌无敌i

在一些应用程序,处理大图像,而不OOM也迅速是非常重要的。

On some apps, it is important to handle large images without OOM and also quickly.

对于这一点,JNI(或renderscript,这可悲的是缺乏对文档)可以是一个很好的解决方案。

For this, JNI (or renderscript, which sadly lacks on documentation) can be a nice solution.

在过去,我已经使用JNI旋转巨大的位图,同时避免OOM(链接here ,here和here).这是一个不错的(但烦人硬)的经验,但最终它的工作。

In the past, i've succeeded using JNI for rotating huge bitmaps while avoiding OOM (link here , here and here). it was a nice (yet annoyingly hard) experience, but in the end it worked.

Android框架的许多功能来处理位图,但我不知道什么是对的JNI侧的情况。

the android framework has plenty of functions to handle bitmaps, but i have no idea what is the situation on the JNI side.

我已经知道如何从Android的Java世界的位图传递给JNI世界和背部。

I already know how to pass a bitmap from android's "java world" to the "JNI world" and back.

我不知道是哪些功能,我可以使用的JNI一边帮我处理位图。

What i don't know is which functions I can use on the JNI side to help me with bitmaps.

我希望能够做的JNI所有的图像操作(包括解码),这样我就不用担心OOM时presented大图像,并在这一过程结束时,我可以将数据转换成Java的位图(显示用户)和/或写入到一个文件中。

I wish to be able to do all image operations (including decoding) on JNI, so that I won't need to worry about OOM when presented with large images, and in the end of the process, I could convert the data to Java-bitmap (to show the user) and/or write it to a file.

再次,我不想把数据转换的JNI侧到Java位图只是为了能够运行这些操作。

again, i don't want to convert the data on the JNI side to a java bitmap just to be able to run those operations.

事实证明,有一些库,提供了许多功能(如 JavaCV ),但他们是相当大的,我不太肯定自己的特色,如果他们真的做的JNI端解码,所以我会preFER要能知道通过内置的Andr​​oid的JNI函数。

As it turns out, there are some libraries that offer many functions (like JavaCV), but they are quite large and I'm not quite sure about their features and if they really do the decoding on the JNI-side, so I would prefer to be able to know what is possible via the built-in JNI function of Android instead.

这可用于图像处理的JNI端在Android?

which functions are available for image manipulation on the JNI side on android?

例如,我怎么能运行在位图的人脸检测,应用矩阵,下取样位图,位图规模,等等...?

for example, how could i run face detection on bitmaps, apply matrices, downsample bitmaps, scale bitmaps, and so on... ?

有关的一些操作,我已经可以想办法来实现它们(缩放图像是相当容易的,维基百科可以帮助很多),但也有一些非常复杂的。

for some of the operations, i can already think of a way to implement them (scaling images is quite easy, and wikipedia can help a lot), but some are very complex.

就算我由我自己执行的操作,也许其他人已经使它更有效,在如此多的优化思想的C / C ++可以有。

even if i do implement the operations by myself, maybe others have made it much more efficiently, thinking of the so many optimizations that C/C++ can have.

去到Android,在那里我需要从头开始实现everythign的JNI侧时,我真的对我自己?

am i really on my own when going to the JNI side of android, where i need to implement everythign from scratch?

只是要清楚,我所感兴趣的是:

just to make it clear, what i'm interested in is:

Java的输入位 - >图像处理纯粹是JNI和C / C ++(无皈依到Java对象的任何) - > Java的输出位图

input bitmap on java -> image manipulation purely in JNI and C/C++ (no convertion to java objects whatsoever) ->output bitmap on java.

推荐答案

内置Android的JNI函数是一种自相矛盾的。这是技术上是正确的,许多Android框架的Java类使用JNI在这条链要调用本机库。

"built-in JNI function of Android" is kind of oxymoron. It's technically correct that many Android Framework Java classes use JNI somewhere down the chain to invoke native libraries.

但也有关于这说明三保留。

But there are three reservations regarding this statement.

这些都是实施细则,并随时更改,恕不另行通知任何下一版本的Andr​​oid或任何叉(如Kindle的),甚至是OEM版本,它不被视为一个的叉(如三星建,或Quallcom SOC)

These are "implementation details", and are subject to change without notice in any next release of Android, or any fork (e.g. Kindle), or even OEM version which is not regarded a "fork" (e.g. built by Samsung, or for Quallcom SOC).

本机方法是在核心Java类中实现的方式是从经典JNI不同。这些方法是preloaded和由JVM缓存,因此不会从大部分的开销典型的JNI调用受到影响。

The way native methods are implemented in core Java classes is different from the "classical" JNI. These methods are preloaded and cached by the JVM and are therefore do not suffer from most of the overhead typical for JNI calls.

没有什么Java或本地code可以做到直接与其他类构成制度框架的JNI方法,尤其是类进行交互。

There is nothing your Java or native code can do to interact directly with the JNI methods of other classes, especially classes that constitute the system framework.

这一切,说,你可以自由地学习的Andr​​oid 源$ C ​​$ C,发现备份特定的本地库类和方法(如人脸检测),并在您的本地code使用这些库,或建立一个JNI层自己的从Java code使用这些库。

All this said, you are free to study the source code of Android, to find the native libraries that back specific classes and methods (e.g. face detection), and use these libraries in your native code, or build a JNI layer of your own to use these libraries from your Java code.

要给出一个具体的例子,人脸检测在Android中通过android.media.FaceDetector类,它加载 libFFTEm.so 。你可以看一下在本土code ,并把它作为你的愿望。你不应该假设 libFFTEm.so 将在设备上present,或在设备库将具有相同的API。

To give a specific example, face detection in Android is implemented through the android.media.FaceDetector class, which loads libFFTEm.so. You can look at the native code, and use it as you wish. You should not assume that libFFTEm.so will be present on the device, or that the library on device will have same API.

但在这种特殊情况下,这不是一个问题,因为内文所有的工作是完全基于软件。因此,您可以复制此code的全部内容,或仅相关的部分,并让它成为你的本机库的一部分。请注意,许多设备,你可以简单地加载并使用 /system/lib/libFFTEm.so 并不会觉得不舒服,直到你遇到一个系统,将行为不端。

But in this specific case, it's not a problem, because all work of neven is entirely software based. Therefore you can copy this code in its entirety, or only relevant parts of it, and make it part of your native library. Note that for many devices you can simply load and use /system/lib/libFFTEm.so and never feel discomfort, until you encounter a system that will misbehave.

一个值得注意的结论,你可以从阅读本土code 是,底层算法忽略颜色信息。因此,如果您要查找的脸坐标图像来自YUV源,就能避免大量的开销,如果你调用

One noteworthy conclusion you can make from reading the native code, is that the underlying algorithms ignore the color information. Therefore, if the image for which you want to find face coordinates comes from YUV source, you can avoid a lot of overhead if you call

// run detection
btk_DCR_assignGrayByteImage(hdcr, bwbuffer, width, height);

int numberOfFaces = 0;
if (btk_FaceFinder_putDCR(hfd, hdcr) == btk_STATUS_OK) {
    numberOfFaces = btk_FaceFinder_faces(hfd);
} else {
    ALOGE("ERROR: Return 0 faces because error exists in btk_FaceFinder_putDCR.\n");
}

直接与YUV(或Y)字节数组,而不是将其转换为RGB和回到YUV在android.media.FaceDetector.findFaces().如果您的YUV缓冲来自于爪哇,你可以建立自己的类 YuvFaceDetector 这将是android.media.FaceDetector与的唯一区别YuvFaceDetector.findFaces()将Y(亮度)值,而不是仅仅一个位图,并避免RGB到Y的变换。

directly with your YUV (or Y) byte array, instead of converting it to RGB and back to YUV in android.media.FaceDetector.findFaces(). If your YUV buffer comes from Java, you can build your own class YuvFaceDetector which will be a copy of android.media.FaceDetector with the only difference that YuvFaceDetector.findFaces() will take Y (luminance) values only instead of a Bitmap, and avoid the RGB to Y conversion.

另外一些情况是不是因为这容易。例如,视频codeCS是紧耦合的硬件平台,并从libstagefright.so为您的项目。 JPEG codeC是一个特殊的野兽。在现代系统(IIRC,因为2.2),你可以期望 /system/lib/libjpeg.so 是present。但是很多平台也有JPEG格式codeCS更有效的硬件实现通过 libstagefright.so 或的OpenMAX,而且往往这些在android.graphics.Bitmap.com$p$pss()和android.graphics.BitmapFactory.de$c$c***()的方法。

Some other situations are not as easy as this. For example, the video codecs are tightly coupled to the hardware platform, and you cannot simply copy the code from libstagefright.so to your project. Jpeg codec is a special beast. In modern systems (IIRC, since 2.2), you can expect /system/lib/libjpeg.so to be present. But many platforms also have much more efficient HW implementations of Jpeg codecs through libstagefright.so or OpenMAX, and often these are used in android.graphics.Bitmap.compress() and android.graphics.BitmapFactory.decode***() methods.

和存在也是一个优化的的libjpeg涡轮增压,该过 /system/lib/libjpeg.so 。

And there also is an optimized libjpeg-turbo, which has its own advantages over /system/lib/libjpeg.so.