OpenGL/DirectX Hook - Similar to FRAPS

Question

Is it possible to detect what applications are using OpenGL or DirectX similar to what FRAPS does? (Possibly using some form of hook)? I probably won't need to actually draw to the window, I just need to know what processes are doing some form of 3D rendering for the time being.

(Edit:) In case you are not familiar with it, FRAPS is a program that can be used to draw a "Frame-per-second" counter on a 3D application. FRAPS finds all running 3D applications by itself without needing you to specify the process name.

Example of "Frame Per second" counter drawn to external game:

Solution 1

Probably the simplest way is to check for the presence of the OpenGL and DirectX core libraries, probably also a good idea to add in the driver OGL dlls in too (such as nvogl), this can be done via EnumProcesses & EnumProcessModulesEx, using p/invoke, this will at least give you a starting set of processes possibly using OGL or DX.

Of course some applications load both of the API's and use only one, or only conditionally use one of the GFX API's (though the latter only occurs with specialized tools and the like), for this, IMO, the best way to check is to perform some form of injection or attaching to the process like a debugger would, then hooking either Present for DX or wglSwapBuffers for OGL.

You might be about to get away with not using a hook by enumerating the GDI handles and looking for the DXGI or OGL render contexts, how viable this is, I don't know.

Solution 2

From what I understand, FRAPS uses a relatively brute force approach to determining where to lay down shop. The process gets started with SetWindowsHookEx to request that the OS load the FRAPS hook DLL into every running process it can [and future processes]. The magic in the DLL comes down to running a procedural set of tests using GetModuleHandleA to observe if the process it is attached to has loaded any OpenGL/DirectX modules. If all calls return NULL, the hook attempts to remove itself from the process.

On the other hand, if the process has loaded them, it simply hooks the appropriate rendering function from that library by removing protection and injecting a JMP hook. wglSwapBuffers is typically the only relevant one in OpenGL. When the process calls this function, it ends up calling the FRAPS module and then FRAPS captures the back buffer into its queue for encoding to AVI and renders its little indication. Then it processes the original request for wglSwapBuffers and returns the execution back to the program.

As far as querying in C#... review EasyHook (http://easyhook.codeplex.com/) and see if it doesn't work for you. I personally have no experience with this API.

Solution 3

You should take a look at How to overlay graphics on Windows games? and the linked article http://www.ring3circus.com/gameprogramming/case-study-fraps/.

The second link has some concept code that should do the actual hook part. It also goes into some greater detail about how and why than I feel like copying and pasting into this answer.

Comments

[Deanna]

Be careful that not all application using DirectX/Direct3d are doing 3D rendering. We use D3D for rendering of H.264 video.

[Zilog]

I think this can be helpful for dx ... Hooking DirectX EndScene from an injected DLL

[David]

From MSDN: "If this function is called from a 32-bit application running on WOW64, it can only enumerate the modules of a 32-bit process. If the process is a 64-bit process, this function fails and the last error code is ERROR_PARTIAL_COPY (299)." This wasn't in the question, but I need something which works on both 32/64bit. What if the process is 64 bit? Does this mean that if I compile the app to 64 bit it should work with 32/64? Or only that if you are 32 bit you can't enumerate 64 bit apps?

[Necrolis]

@David: there is a 32 & 64 bit variant available(EnumProcessModulesEx), you'll need a 64bit app to get both 32 and 64 bit processes however (see remarks), and you'll still need a 32bit build for 32bit OSes.