Does it make sense to create canvas-based UI components?

Question

While DOM still totally dominates the way we create UIs, does it make sense to create a bunch of entirely canvas-based UI components, like buttons, lists, horizontal/vertical groups, etc?

I know for sure that there will be a lot of drawbacks, but what would the possible advantages of such be?

For one, I'd say in general the visual integration with canvas will be much tighter.

Solution 1

Yes and no.

In terms of accenting the UI of the page, it's somewhat ideal if the page navigation and operation can also work well if JavaScript is not enabled.

It is okay to have UI elements that are prettified by canvas, but remember that these elements will not be accessible to web crawlers (like Google), or to users who have scripts disabled.

No:

Don't try to remake text editing in canvas. Even the HTML Canvas Spec has a section where they strongly advise against trying to create text-editing controls in Canvas.

There's a long history of trying to do that and failing (look up Mozilla Bespin)

I know for sure that there will be a lot of drawbacks, but what would the possible advantages of such be?

Yes:

Visualizations (that have no fallback except text description) and controls like dials (that fall back to some HTML, like input type="range" sliders) can be greatly enhanced with Canvas.

If you search for canvas controls, canvas diagrams, canvas charts, etc you'll find a lot of libraries that offer stuff like this. Just understand very well that many of these come at the expense of accessibility.

Maybe:

There are a lot of pretty elements you can add to a page with Canvas. Some things can get really beautiful without being intrusive or altering the page navigation in any way. Perhaps the logo of a website would "grow" procedurally or glow or otherwise get more complex. Other background animation effects might be really neat.

There are also interactive images, like on sites where you want a diagram or breakdown or exploded view that you would navigate to inspect the various parts of something (a chemical structure, a biological organism, a new car). Visual interactive media such as diagramming and games are some of the best use-cases for Canvas.

Solution 2

The Zebra project has created a full component set which is rendered into a HTML5 canvas element. Here is a screenshot of the component sampler. I haven't used the framework, but it should give you an idea of how well the different browsers can render UI components.

Rotate the components and check the quality of line rendering (anti-aliasing), which is very different depending on the browser you use. Here's some more information regarding that problem:

• HTML5 Canvas avoid any subpixel rendering
• Poor anti-aliasing of text drawn on Canvas
• Subpixel anti-aliased text on HTML5's canvas element

Another project is Makepad, a webGL worker-based library and live code editor. Every visible part of the UI is rendered in WebGL, including all text on screen, rendered through the integrated text rendering engine.

It is still early phase for the project, but you can try out a live demo here. Makepad is open source, the Git repo can be found at github.com/makepad/makepad.github.io.

Solution 3

Using Canvas as a UI base is an excellent idea if you have > 200 elements. It's much, much faster to render than using DOM elements.

On iPhone Safari, 300 animated DOM elements runs at 3fps (frames per second), very slow.

If you use canvas, you can render > 300 elements and still achieve 30fps, which means smooth animation and transitions. I've tested this at length so I know it works.

The downside to Canvas (as someone else mentioned), is that search engines can't see your content. But if you are building an app that shouldn't be spidered and needs to run on mobile, then Canvas is the way to go.

Solution 4

For the last four years I have been building components for the canvas including buttons, dials, sliders, check boxes, radio buttons, color pickers, panes, windows, indicators, waiters, steppers, tabs, pads, etc. see http://zimjs.com/components/ for working examples.

The advantages are as follows:

1. The components can be customized in more or different ways than traditional HTML/CSS components and we can make more types of components. See http://zimjs.com/docs.html for examples.
2. It is often important when making interactive works to embed the components right in the app or game. This is difficult to do by overlaying HTML components when taking into account scaling apps.
3. There is definitely tighter integration when dragging panels, animating components, scaling components, working with canvas library events (ZIM and CreateJS use on() method which has benefits - canvas components can make use of this).

I love working with Canvas components - it saves lines of code and I don't have to switch between systems. Just a reminder... the format of CSS is basically the same as an object literal in code. I would rather format my components in code any day rather than CSS - personally, I find it much easier to work in one system.

In terms of screen reader results for interface - many canvas creations are not suited for visually impaired. It can still be done, as pointed out, if applicable.

One final comment... consistency is an important design principle but variety is the spice of life. I do not think we should be relying on a homogeneous interface system. There should be room for growth, experimentation and exploration.

Solution 5

We've tried something like this but finally came up with the idea that the world is not ready yet )

You should keep in mind following

• js always should be enabled. Nowadays one can consider it not a big deal, but nevertheless it worth to mention.
• html/css is actually traditional and constantly evolving stack of standards, sooner or later you'll feel the need in having some descriptive language to reduce repeating code in your canvas rendered UI-components. And there are two options here - to try to invent something proprietary, which actually could be fun and interesting, but can have some very sad consequences. The second way is to reimplement html/css not to confuse third party developers. But, wait a minute, we've already have html/css engine )))
• events and, therefore, user experience. Jonas is right. Trying to reimplement even a subset of js event model to make it more comfortable to develop canvas rendered components is hard. Some issues even are unsolvable.

So, it is actually interesting experience, but I would not recommend.

View more solutions

Comments

[Tak]

This reminds me of the fashion a few years ago to create navigation buttons in Java. There were a small few advantages but the disadvantages were huge. IMO it's not a good idea.

[Alex K]

@Tak ... this was a thing that people actually... did? And they thought it was a good idea?!!

[Werner]

May be not a problem at all but I guess people which have to use screenreaders would not be able to use the page.

[Nikola Lukic]

Try bitbucket.org/nikola_l/visual-js , you can found online examples ...

[Berislav Lopac]

Actually, while I agree with most of your assertions, I would argue that this is not such a clear cut in some circumstances. For example, if you don't need your UI to be globally accessible (e.g. for a purely intranet-facing app), and of course in the cases mentioned by nick fallon in his answer (if you have a lot of items and have to run on mobile) -- and especially if those circumstances combine (as in my case, where we have what amounts to an in-browser spreadsheet with potentially thousands of cells, running strictly internally on the local network), canvas is definitely a viable option.

[trusktr]

Who says it's not indexable? Put all your meta data in some display:none elements, then make your entire ui in canvas, and leverage WebGL. Totally doable while maintaining indexability.

[Josh]

What about a canvas game that needs menu buttons?

[Simon Sarris]

Often the game HUD/menus are done separately from the canvas, which makes internationalization a little easier, but I've seen it both ways. @AshleysBrain do you want to weigh in?

[jperelli]

How about apps that doesn't require google crawling. For example, take a look at the implementation behind google collaborative docs. There is no way you can show two carets on one input text, the only way to do it is to implement a canvas super control.

[Resist Design]

The future will show that this answer is wrong. It will take time but the persisting diversity among browser implementations of DOM and CSS are about to finally drive every toward a more unified UI system. Flex, the UI component framework for Flash, existed because HTML had already failed once before, for this very reason. Now that we have canvas, it is just a matter of time until people get their heads on straight again. -1 for ignoring history.

[Alan B]

The two billion Javascript UI libraries, frameworks and toolkits.

[Alan B]

Who cares if search engines can't see your content in the context of how this sort of UI would be deployed?

[Pacerier]

Good post. The advantage of self-rendering is potentially faster performance because the browser is built to be optimized for the general use case.

[Jonas Sandstedt]

The URL is broken. The new URL is html.spec.whatwg.org/multipage/scripting.html#best-practices

[Dan Zen]

Since the time of the response above, ZIM has added Accessibility for all its canvas components. As this is not directly supported by a single bitmap canvas, the support is by placing HTML tags in behind the canvas. zimjs.com/accessibility But this does give an example of how Accessibility and canvas components can work.

[Totti]

How would you write a WYSIWYG editor for typeset mathematics in Javascript?

[slumtrimpet]

Looks interesting, but... the product page is a bit offputting. Here's a quote: "Digital advertising has surpassed traditional media buys. ZIM provides easy interaction and animation and can often be distilled to 40k to keep ad sizes low. See MORE for ideas and tips..." Those are in fact words there, but they are meaningless. The entire product page seems to be full of snips like that to the extent that it actually gave me a headache.

[slumtrimpet]

PS... upvoting anyway because I really do like this concept, just need to mellow the product page a bit to make it less chaotic feeling.

[Dan Zen]

Thanks for the note @slumtrimpet. That description is from one of ten types of features where coding on the Canvas makes sense - in this case interactive advertising (admusements). These sections are somewhat promotional - compared to the rest of the site - sorry they gave you a headache and thanks for the upvote.

[Stas Gavrylov]

Amazing work! I agree that we need to experiment with interfaces, and I hope to see an increase of such projects in future.