How to inspect Canvas Frames

Question

I just saw this link, about how to inspect "Canvas frames" on Chrome Dev Tools, but how to do it on Chrome extensions?

I think that the chrome.debugger.sendCommand method should be used here.

How to do it on Chrome Extensions?

EDIT: I usually want to inspect these data using Chrome's debugger or some similar method, without interacting with the JavaScript's code.

Solution 1

The experimental Canvas inspection feature was removed from Chrome 44 (crbug.com/475808). It was implemented by capturing calls to canvas methods in the page.

I previously used this Canvas inspection feature to create reduced test cases for <canvas> bugs. Upon discovering the removal of the Canvas inspection feature, I developed a new tool to capture snapshots of the canvas. This tool intercepts all canvas commands and stores the result in the 2D context, and also offers a method to retrieve all commands played thus far. The tool and documentation are available at https://github.com/Rob--W/canvas-interceptor.

Currently, only a subset of the 2D canvas APIs are captured, because not every kind of parameter is serialized yet. Everything that is not implemented is marked as TODO in the source code, so if you want to build upon this tool, feel free to submit pull requests to fill in the missing parts.

To use this tool in an extension, inject the code in the page via a content script with run_at document_start. Since saving the canvas state is quite expensive in terms of memory, I recommend to only insert the script when needed by using the declarativeContent.RequestContentScript action.

Solution 2

EDIT: As shown in RobW's answer, required functionality was removed from Chrome.

The relevant documentation is at tip-of-the-tree debugging protocol docs.

Specifically, this is done via the LayerTree domain. Do note: since it's not in an officially supported version of the debugger protocol, it's subject to change without notice.

The flow with the API presumably would be to attach to the page, send a command to enable profiling and then listen for appropriate events.

You can see a sample of how Debugger works, and try to figure it out from the docs or sniff the actual protocol by remote-debugging a Dev Tools page's WebSocket traffic.

Comments

[Walter Chapilliquen - wZVanG]

any idea to do it on background?

[Xan]

The idea is given in the answer. I don't have actual code. A combination of reading the docs and reverse-engineering the protocol through remote debugging should give you the idea of which commands/events are exchanged.

[Walter Chapilliquen - wZVanG]

Thank you. I tried this method LayerTree.enable with this simple canvas and the results are: http://i.imgur.com/HwLxtjW.png Now, in this case, how can to recompile canvas.drawImage changes?

[Xan]

LayerTree.replaySnapshot should do it.

[Walter Chapilliquen - wZVanG]

Thanks, replaySnapshot returns dataUrl image, but I want to see how to get the parameters of drawImage(img,x,y,w,h) call. In devtools: i.imgur.com/WUCaQif.png

[Walter Chapilliquen - wZVanG]

Hi @Xan, there will be some way?

[Walter Chapilliquen - wZVanG]

+1 For the interceptor canvas, but that's a code injection, so I do not want any way to inject some code into the page context, and the site should not know that there a code injection. I usually want to inspect these data using Chrome's debugger or some similar method, without interacting with the JavaScript's code

[Rob W]

@wZVanG The (now removed) Canvas inspection feature also used code injection. You can design the functionality in such a way that it is virtually undetectable by the page (similar to your question about WebSockets - stackoverflow.com/a/31182643), but that takes lots of efforts. If you really want to be undetectable to the JavaScript code of the page, you have to compile a custom version of Chrome or Firefox.

[Walter Chapilliquen - wZVanG]

That's exactly what I want to do, make a customized version of Chrome, making my needs could be resolved in general. Users will not have problem to download a version of Chrome. For example, currently my users must install Chrome Dev. I know that is not the question, but you could give me an idea of how to start do it?

[Rob W]

@wZVanG I've posted instructions to build a specific version of Chromium at stackoverflow.com/a/26462483/938089. See also chromium.org/developers/how-tos/api-keys and chromium.org/developers/how-tos/build-instructions-windows. For Firefox, see developer.mozilla.org/en/docs/Simple_Firefox_build and developer.mozilla.org/en/docs/Introduction.

[vals]

Around the years, I have found tools to debug OpenGL - WebGL... only to loose them some time later.... This one was pretty cool , it's very frustating to have lost it