Drawing UI elements directly to the WebGL area with Three.js

Question

In Three.js, is it possible to draw directly to the WebGL area (for a heads-up display or UI elements, for example) the way you could with a regular HTML5 canvas element?

If so, how can you get the context and what drawing commands are available?

If not, is there another way to accomplish this, through other Three.js or WebGL-specific drawing commands that would cooperate with Three.js?

My backup plan is to use HTML divs as overlays, but I think there should be a better solution.

Thanks!

Solution 1

You can't draw directly to the WebGL canvas in the same way you do with with regular canvas. However, there are other methods, e.g.

• Draw to a hidden 2D canvas as usual and transfer that to WebGL by using it as a texture to a quad
• Draw images using texture mapped quads (e.g. frames of your health box)
• Draw paths (and shapes) by putting their vertices to a VBO and draw that with the appropriate polygon type
• Draw text by using a bitmap font (basically textured quads) or real geometry (three.js has examples and helpers for this)

Using these usually means setting up a an orthographic camera.

However, all this is quite a bit of work and e.g. drawing text with real geometry can be expensive. If you can make do with HTML divs with CSS styling, you should use them as it's very quick to set up. Also, drawing over the WebGL canvas, perhaps using transparency, should be a strong hint to the browser to GPU accelerate its div drawing if it doesn't already accelerate everything.

Also remember that you can achieve quite much with CSS3, e.g. rounded corners, alpha transparency, even 3d perspective transformations as demonstrated by Anton's link in the question's comment.

Solution 2

I had exactly the same issue. I was trying to create a HUD (Head-up display) without DOM and I ended up creating this solution:

1. I created a separate scene with orthographic camera.
2. I created a canvas element and used 2D drawing primitives to render my graphics.
3. Then I created an plane fitting the whole screen and used 2D canvas element as a texture.
4. I rendered that secondary scene on top of the original scene

That's how the HUD code looks like:

// We will use 2D canvas element to render our HUD.  
var hudCanvas = document.createElement('canvas');

// Again, set dimensions to fit the screen.
hudCanvas.width = width;
hudCanvas.height = height;

// Get 2D context and draw something supercool.
var hudBitmap = hudCanvas.getContext('2d');
hudBitmap.font = "Normal 40px Arial";
hudBitmap.textAlign = 'center';
hudBitmap.fillStyle = "rgba(245,245,245,0.75)";
hudBitmap.fillText('Initializing...', width / 2, height / 2);

// Create the camera and set the viewport to match the screen dimensions.
var cameraHUD = new THREE.OrthographicCamera(-width/2, width/2, height/2, -height/2, 0, 30 );

// Create also a custom scene for HUD.
sceneHUD = new THREE.Scene();

// Create texture from rendered graphics.
var hudTexture = new THREE.Texture(hudCanvas) 
hudTexture.needsUpdate = true;

// Create HUD material.
var material = new THREE.MeshBasicMaterial( {map: hudTexture} );
material.transparent = true;

// Create plane to render the HUD. This plane fill the whole screen.
var planeGeometry = new THREE.PlaneGeometry( width, height );
var plane = new THREE.Mesh( planeGeometry, material );
sceneHUD.add( plane );

And that's what I added to my render loop:

// Render HUD on top of the scene.
renderer.render(sceneHUD, cameraHUD);

You can play with the full source code here: http://codepen.io/jaamo/pen/MaOGZV

And read more about the implementation on my blog: http://www.evermade.fi/pure-three-js-hud/

Comments

[Dev]

great answer, thanks.. also see comments above as Anton & WestLangley had some good insight

[trusktr]

hmmmm...... I wonder if making the entire background of my page an empty webgl canvas will force the browser to accelerate everything based on your comment. For some some reason I don't think it'll work.

[junvar]

Thanks for a clear example. Though I would recommend using OffscreenCanvas (chrome m69+) instead of document.createElement('canvas')