How to record screen and take screenshots, using Android API?

Question

Background

Android got a new API on Kitkat and Lollipop, to video capture the screen. You can do it either via the ADB tool or via code (starting from Lollipop).

Ever since the new API was out, many apps came to that use this feature, allowing to record the screen, and Microsoft even made its own Google-Now-On-tap competitor app.

Using ADB, you can use:

adb shell screenrecord /sdcard/video.mp4 

You can even do it from within Android Studio itself.

The problem

I can't find any tutorial or explanation about how to do it using the API, meaning in code.

What I've found

The only place I've found is the documentations (here, under "Screen capturing and sharing"), telling me this:

Android 5.0 lets you add screen capturing and screen sharing capabilities to your app with the new android.media.projection APIs. This functionality is useful, for example, if you want to enable screen sharing in a video conferencing app.

The new createVirtualDisplay() method allows your app to capture the contents of the main screen (the default display) into a Surface object, which your app can then send across the network. The API only allows capturing non-secure screen content, and not system audio. To begin screen capturing, your app must first request the user’s permission by launching a screen capture dialog using an Intent obtained through the createScreenCaptureIntent() method.

For an example of how to use the new APIs, see the MediaProjectionDemo class in the sample project.

Thing is, I can't find any "MediaProjectionDemo" sample. Instead, I've found "Screen Capture" sample, but I don't understand how it works, as when I've run it, all I've seen is a blinking screen and I don't think it saves the video to a file. The sample seems very buggy.

The questions

How do I perform those actions using the new API:

1. start recording, optionally including audio (mic/speaker/both).
2. stop recording
3. take a screenshot instead of video.

Also, how do I customize it (resolution, requested fps, colors, time...)?

Solution

First step and the one which Ken White rightly suggested & which you may have already covered is the Example Code provided officially.

I have used their API earlier. I agree screenshot is pretty straight forward. But, screen recording is also under similar lines.

I will answer your questions in 3 sections and will wrap it up with a link. :)

---

1. Start Video Recording

private void startScreenRecord(final Intent intent) {
 if (DEBUG) Log.v(TAG, "startScreenRecord:sMuxer=" + sMuxer);
 synchronized(sSync) {
  if (sMuxer == null) {
   final int resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0);
   // get MediaProjection 
   final MediaProjection projection = mMediaProjectionManager.getMediaProjection(resultCode, intent);
   if (projection != null) {
    final DisplayMetrics metrics = getResources().getDisplayMetrics();
    final int density = metrics.densityDpi;

    if (DEBUG) Log.v(TAG, "startRecording:");
    try {
     sMuxer = new MediaMuxerWrapper(".mp4"); // if you record audio only, ".m4a" is also OK. 
     if (true) {
      // for screen capturing 
      new MediaScreenEncoder(sMuxer, mMediaEncoderListener,
       projection, metrics.widthPixels, metrics.heightPixels, density);
     }
     if (true) {
      // for audio capturing 
      new MediaAudioEncoder(sMuxer, mMediaEncoderListener);
     }
     sMuxer.prepare();
     sMuxer.startRecording();
    } catch (final IOException e) {
     Log.e(TAG, "startScreenRecord:", e);
    }
   }
  }
 }
}

2. Stop Video Recording

 private void stopScreenRecord() {
  if (DEBUG) Log.v(TAG, "stopScreenRecord:sMuxer=" + sMuxer);
  synchronized(sSync) {
   if (sMuxer != null) {
    sMuxer.stopRecording();
    sMuxer = null;
    // you should not wait here 
   }
  }
 }

2.5. Pause and Resume Video Recording

 private void pauseScreenRecord() {
  synchronized(sSync) {
   if (sMuxer != null) {
    sMuxer.pauseRecording();
   }
  }
 }

 private void resumeScreenRecord() {
  synchronized(sSync) {
   if (sMuxer != null) {
    sMuxer.resumeRecording();
   }
  }
 }

Hope the code helps. Here is the original link to the code that I referred to and from which this implementation(Video recording) is also derived from.

---

3. Take screenshot Instead of Video

I think by default its easy to capture the image in bitmap format. You can still go ahead with MediaProjectionDemo example to capture screenshot.

[EDIT] : Code encrypt for screenshot

a. To create virtual display depending on device width / height

mImageReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 2);
mVirtualDisplay = sMediaProjection.createVirtualDisplay(SCREENCAP_NAME, mWidth, mHeight, mDensity, VIRTUAL_DISPLAY_FLAGS, mImageReader.getSurface(), null, mHandler);
mImageReader.setOnImageAvailableListener(new ImageAvailableListener(), mHandler);

b. Then start the Screen Capture based on an intent or action-

startActivityForResult(mProjectionManager.createScreenCaptureIntent(), REQUEST_CODE);

Stop Media projection-

sMediaProjection.stop();

c. Then convert to image-

//Process the media capture
image = mImageReader.acquireLatestImage();
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * mWidth;
//Create bitmap
bitmap = Bitmap.createBitmap(mWidth + rowPadding / pixelStride, mHeight, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
//Write Bitmap to file in some path on the phone
fos = new FileOutputStream(STORE_DIRECTORY + "/myscreen_" + IMAGES_PRODUCED + ".png");
bitmap.compress(CompressFormat.PNG, 100, fos);
fos.close();

---

There are several implementations (full code) of Media Projection API available. Some other links that can help you in your development-

1. Video Recording with MediaProjectionManager - website

2. android-ScreenCapture - github as per android developer's observations :)

3. screenrecorder - github

4. Capture and Record Android Screen using MediaProjection APIs - website

---

Hope it helps :) Happy coding and screen recording!

PS: Can you please tell me the Microsoft app you are talking about? I have not used it. Would like to try it :)

Comments

[android developer]

Microsoft made Cortana for Android. After some reading, I think they didn't use screen capture, but I'm not sure. For Android 6, they probably used what Android has to offer already.

[bozzmob]

Oh ok! I have used Cortana and its not based on this API mostly. They have several other "Services" which capture information. Same is the case with Arrow launcher and Yahoo Aviate. Anyway, hope my answer helped :)

[android developer]

Are there any limitations about the video capture features that I should know about ? This sample doesn't do anything on my case: github.com/googlesamples/android-ScreenCapture , but this one works: github.com/chinmoyp/screenrecorder . How do I capture a single screenshot?

[bozzmob]

I have pointed out on how to take screenshots- github.com/mtsahakis/MediaProjectionDemo . This demo has it.

[bozzmob]

Limitation-wise, I don't see anything much there. But, when it comes to the app-wise, I can say, if you start recording video on a 1920x1080 resolution phone(in my case and most others), the size of the video grows up like bacteria. So, phones with lesser memory should be handled well. API wise, its not too resource intensive.

[android developer]

About limitations, ok. about screenshot, please show code for this, instead of a repo. i know there is a tiny chance for it to get removed, but still, a clean and small solution should also be available here... :)

[bozzmob]

Done! Added most relevant code & steps needed get image out of Media Projection APIs. Hope it helps :)

[android developer]

ok you deserve the bounty and +1 for all the effort. Thank you, and if you have anything more to write about this, please do.

[bozzmob]

Yes indeed :) With time, if there are anything that I come across, I will update.

[android developer]

I think the code is missing some lines. For example, what's with the intent that you have for "startScreenRecord" ? Where is "REQUEST_CODE" being used?

[Ewoks]

why one should take screenshot like this instead of using setDrawingCacheEnabled(true); and than getDrawingCache() ? is it more cpu/memory efficient or there is some other reason?

[Sagar Panwala]

Does screen record with/without audio work below Lollipop devices? If no then what is the alternate of solution?

[android developer]

@Ewoks Screenshot works globally, taking a picture of what the user sees, including of other apps. What you wrote will work only within the current app.

[android developer]

@bozzmob It seems this code causes transparent margins on left and right. I've created a new post about it here: stackoverflow.com/q/43705756/878126

[Admin]

can't we capture screenshot without startActivityForResult i mean in service lets say i receive screenshot string from server and it captured automatically ?

[bozzmob]

@Eazyz I'm not sure of that. Will let others to comment.

[android developer]

@Eazyz No, but you can make it look like you did it this way: stackoverflow.com/a/32849121/878126. And, you can request for the permission once per app launch, as written here: stackoverflow.com/a/33892817/878126

[Admin]

@androiddeveloper the problem that my application is always running background . on first installation it works but when the user restart the phone the appliaiton restarts with broadcast and (resultCode-Intent) equals null after that. (i get command from server)

[android developer]

@Eazyz Have you tried starting an Activity and then request it ?

[Admin]

it worked for me thanks for replying me @androiddeveloper

[Admin]

my service starts the screen recoder and working but why always i need to send new request for single shot picture and how to send the request from service I don't have any Activity just first main-activity works once then all things happens with server connection and service background app

[TheRealChx101]

Why do they make these APIs so complicated?

[silentsudo]

see android lifecycle look super awesome, people till today fixing it in their libraries :D

[android developer]

Any idea how to record the video with audio? I think it's possible now on Android Q

[Tarun]

@bozzmob How I can achieve if my application is always in background ?

[Ahmad]

if i use it with floating button am i able to take screenshot of android desktop as well ?