How to take a screenshot of desktop fast with Java in Windows (ffmpeg, etc.)?

java windows ffmpeg java-native-interface

13,844

Solution 1

Using the built-in Robots class is way easier than other Java libraries and should probably fit your needs.

If you need a smooth video with >= 30fps (more than 30 screenshots per second), you should first try the Robots approach plus performance improvements there using asynchronous storing of the screenshots.

If it doesn't work for you, try using JNA and that is (even though it's more complex) almost guaranteed to work for smooth screen capturing.

Approach with Robots

The robots class is indeed capable of doing what you want, the problem most screen capturing approaches with Robots have is the saving of the screenshots. An approach could look like that: Looping over the captureScreen() method, grabbing the screen into a BufferedImage, convert it to a byte array and save it with an asynchronous file writer to a target file after adding the future reference of your image to the ArrayList to be able to keep going while storing the image data.

// Pseudo code
while (capturing)
{
    grab bufferedImage (screenCapture) from screen
    convert bufferImage to byte array
    start asynchronous file channel to write to the output file
      and add the future reference (return value) to the ArrayList
}

Approach with JNA

Original Question: How to take screenshots fast in Java?

As it is bad practice to just link, I will post the example here:

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferInt;
import java.awt.image.DataBufferUShort;
import java.awt.image.DirectColorModel;
import java.awt.image.Raster;
import java.awt.image.WritableRaster;

import com.sun.jna.Native;
import com.sun.jna.platform.win32.W32API;
import com.sun.jna.win32.W32APIOptions;

public class JNAScreenShot
{

    public static BufferedImage getScreenshot(Rectangle bounds)
    {
        W32API.HDC windowDC = GDI.GetDC(USER.GetDesktopWindow());
        W32API.HBITMAP outputBitmap = GDI.CreateCompatibleBitmap(windowDC, bounds.width, bounds.height);
        try
        {
            W32API.HDC blitDC = GDI.CreateCompatibleDC(windowDC);
            try
            {
                W32API.HANDLE oldBitmap = GDI.SelectObject(blitDC, outputBitmap);
                try
                {
                    GDI.BitBlt(blitDC, 0, 0, bounds.width, bounds.height, windowDC, bounds.x, bounds.y, GDI32.SRCCOPY);
                }
                finally
                {
                    GDI.SelectObject(blitDC, oldBitmap);
                }
                GDI32.BITMAPINFO bi = new GDI32.BITMAPINFO(40);
                bi.bmiHeader.biSize = 40;
                boolean ok = GDI.GetDIBits(blitDC, outputBitmap, 0, bounds.height, (byte[]) null, bi, GDI32.DIB_RGB_COLORS);
                if (ok)
                {
                    GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
                    bih.biHeight = -Math.abs(bih.biHeight);
                    bi.bmiHeader.biCompression = 0;
                    return bufferedImageFromBitmap(blitDC, outputBitmap, bi);
                }
                else
                {
                    return null;
                }
            }
            finally
            {
                GDI.DeleteObject(blitDC);
            }
        }
        finally
        {
            GDI.DeleteObject(outputBitmap);
        }
    }

    private static BufferedImage bufferedImageFromBitmap(GDI32.HDC blitDC, GDI32.HBITMAP outputBitmap, GDI32.BITMAPINFO bi)
    {
        GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
        int height = Math.abs(bih.biHeight);
        final ColorModel cm;
        final DataBuffer buffer;
        final WritableRaster raster;
        int strideBits = (bih.biWidth * bih.biBitCount);
        int strideBytesAligned = (((strideBits - 1) | 0x1F) + 1) >> 3;
        final int strideElementsAligned;
        switch (bih.biBitCount)
        {
            case 16:
                strideElementsAligned = strideBytesAligned / 2;
                cm = new DirectColorModel(16, 0x7C00, 0x3E0, 0x1F);
                buffer = new DataBufferUShort(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            case 32:
                strideElementsAligned = strideBytesAligned / 4;
                cm = new DirectColorModel(32, 0xFF0000, 0xFF00, 0xFF);
                buffer = new DataBufferInt(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            default:
                throw new IllegalArgumentException("Unsupported bit count: " + bih.biBitCount);
        }
        final boolean ok;
        switch (buffer.getDataType())
        {
            case DataBuffer.TYPE_INT:
            {
                int[] pixels = ((DataBufferInt) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            case DataBuffer.TYPE_USHORT:
            {
                short[] pixels = ((DataBufferUShort) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            default:
                throw new AssertionError("Unexpected buffer element type: " + buffer.getDataType());
        }
        if (ok)
        {
            return new BufferedImage(cm, raster, false, null);
        }
        else
        {
            return null;
        }
    }

    private static final User32 USER = User32.INSTANCE;

    private static final GDI32 GDI = GDI32.INSTANCE;

}

interface GDI32 extends com.sun.jna.platform.win32.GDI32
{
    GDI32 INSTANCE = (GDI32) Native.loadLibrary(GDI32.class);

    boolean BitBlt(HDC hdcDest, int nXDest, int nYDest, int nWidth, int nHeight, HDC hdcSrc, int nXSrc, int nYSrc, int dwRop);

    HDC GetDC(HWND hWnd);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, byte[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, short[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, int[] pixels, BITMAPINFO bi, int usage);

    int SRCCOPY = 0xCC0020;
}

interface User32 extends com.sun.jna.platform.win32.User32
{
    User32 INSTANCE = (User32) Native.loadLibrary(User32.class, W32APIOptions.UNICODE_OPTIONS);

    HWND GetDesktopWindow();
}

More information and approaches

See also

Solution 2

According to the official ffmpeg documentation you should be able to keep it pretty cross platform if you make the file parameter passed to the FFmpegFrameGrabber (which is really an input parameter that gets passed down as the -i option to ffmpeg) adhere to the different formats each device expects.

ie:

for Windows: dshow expects -i video="screen-capture-recorder"

for OSX: avfoundation expects -i "<screen device index>":

and for Linux: x11grab expects -i :<display id>+<x>,<y>.

So just passing those values (arguments to -i) to the constructor and setting the format (via setFormat) accordingly should do the trick.

Solution 3

You will need to use JNI or JNA to call some combination of CreateCompatibleBitmap, XGetImage, DirectX or OpenGL to grab a screenshot and then copy some raw bitmap data back to Java. My profiling showed a speed up of about 400% over the Robot class when accessing raw bitmap data on X11. I have not tested other platforms at this time. Some very early code is available here but I haven't had much time to work on it recently.

13,844

Author by

Setsuna

Updated on June 05, 2022

Comments

Setsuna about 2 years
I would like to use java to take a screenshot of my machine using FFMPEG or some other solution. I know linux works with ffmpeg without JNI, but running it in Windows does not work and may require (JNI?) is there any sample of some simple Java class (and anything else necessary) to capture a screenshot runnable in a windows environment? Is there some alternative to FFMPEG? I want to take screenshot at a rate faster than the Java Robot API, which I have found to work at taking screenshots, but is slower than I would like.

I know in Linux this works very fast:
```
import com.googlecode.javacv.*;

public class ScreenGrabber {
    public static void main(String[] args) throws Exception {
        int x = 0, y = 0, w = 1024, h = 768;
        FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(":0.0+" + x + "," + y);
        grabber.setFormat("x11grab");
        grabber.setImageWidth(w);
        grabber.setImageHeight(h);
        grabber.start();

        CanvasFrame frame = new CanvasFrame("Screen Capture");
        while (frame.isVisible()) {
            frame.showImage(grabber.grab());
        }
        frame.dispose();
        grabber.stop();
    }
```
This does not work in windows environment. Am not sure if there is some way I could use this same code, but use javacpp to actually get it working without having to change much of the above code.

Goal is to take screenshots of screen fast, but then stop after it takes a screenshot that is "different", aka. screen changed because of some event like, a window is window closed, etc.
Setsuna almost 10 years

Could you provide some sample that works with Windows? I have been able to use FFMPEG on X11, but it does not work on Windows. I've seen JavaCPP, but no one appears to have used it to any success. If only the Robot were able to be able to take things at a minimum of 30 fps...
Alex Barker almost 10 years

Here is a simple GDI example. There is also a DirectX sample in the same folder.
Jeremy about 3 years

I've pulled your source code and I'm not able to get it to run using visual studio code using g++ compiler. Trying to run this on Windows. Always get the following error fatal error: screencapture.h: No such file or directory #include "screencapture.h"