Processing Android camera frames in real time

Question

I'm trying to create an Android application that will process camera frames in real time. To start off with, I just want to display a grayscale version of what the camera sees. I've managed to extract the appropriate values from the byte array in the onPreviewFrame method. Below is just a snippet of my code:

byte[] pic;
int pic_size;
Bitmap picframe;
public void onPreviewFrame(byte[] frame, Camera c)
{
    pic_size = mCamera.getParameters().getPreviewSize().height * mCamera.getParameters().getPreviewSize().width;
    pic = new byte[pic_size];
    for(int i = 0; i < pic_size; i++)
    {
        pic[i] = frame[i];
    }
    picframe = BitmapFactory.decodeByteArray(pic, 0, pic_size);
}

The first [width*height] values of the byte[] frame array are the luminance (greyscale) values. Once I've extracted them, how do I display them on the screen as an image? Its not a 2D array as well, so how would I specify the width and height?

Solution 1

You can get extensive guidance from the OpenCV4Android SDK. Look into their available examples, specifically Tutorial 1 Basic. 0 Android Camera

But, as it was in my case, for intensive image processing, this will get slower than acceptable for a real-time image processing application. A good replacement for their onPreviewFrame 's byte array conversion to YUVImage:

YuvImage yuvImage = new YuvImage(frame, ImageFormat.NV21, width, height, null);

Create a rectangle the same size as the image.

Create a ByteArrayOutputStream and pass this, the rectangle and the compression value to compressToJpeg():

ByteArrayOutputStream baos = new ByteArrayOutputStream(); yuvimage.compressToJpeg(imageSizeRectangle, 100, baos);

byte [] imageData = baos.toByteArray();

Bitmap previewBitmap = BitmapFactory.decodeByteArray(imageData , 0, imageData .length);

Rendering these previewFrames on a surface and the best practices involved is a new dimension. =)

Solution 2

This very old post has caught my attention now.

The API available in '11 was much more limited. Today one can use SurfaceTexture (see example) to preview camera stream after (some) manipulations.

Comments

[Rekin]

Does this happen to convert the YUV image to a Bitmap with an intermediary JPEG? Hardly a feasible real-time solution

[protectedmember]

I would strongly advise against this. OpenCV on android will add complexity to the project and will also require the native libraries which are around 10mb in size. Android does have all the facilities to accomplish what the OP requires. OpenCV is a great library, but overkill for this situation.