Real headless browser

Question

I am in charge of testing an enterprise web application that makes heavy use of AJAX. I need to build a system that allows continuous running of tests without human intervention. Currently I am mostly interested in load testing but I want the same scripts used to generate load to be used in functional tests.

Currently for loadtesting I am using the grinder. We record scripts and then process the heck out of them to handle among other things the asynchronous requests. This system works for now; however the scripts are delicate and I can't maintain them during our development cycle. I need to be able to 'run a browser' via a programming language that abstracts over the details of html and javascript but it must be headless. In other words if my testing script breaks it must represent an actual breakage of the application. Running firefox in xvfb does not work as firefox still uses too much resources even on a headless linux machine running with xvfb. I tried driving firefox with webdriver under the headless x server.

I have been working on HTMLUnit for days since this is the ideal solution. I have been writing HTMLUnit drivers in Jython and hence I can use them with the grinder. Unfortunately I am running into javascript errors (that are not real errors in firefox/chrome/IE) and I think I am at the end of the road for this. In any case I am well aware of HtmlUnit and I really need an alternative.

I know there are some other solutions (envjs and zombie.js); however I don't know how developed these technologies are and I don't want to waste another week going down a dead end.

How hard would it be to take the source code for either Firefox or Webkit and comment out all the rendering/GUI calls and create a real headless browser? Has this been done? Would one be easier to do this with than another? Honestly I cannot fathom why this has not been done already, so I am guessing it is much harder than I am anticipating.

I am assuming that if I can get a truly headless browser with reasonable performance characteristics (I have a large server fleet to throw at this problem, but its not big enough for real firefox with GUI rendering) then I will be set.

Solution 1

This has been done by the guys over at http://www.phantomjs.org/. In their own words:

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

PhantomJS is an optimal solution for headless testing of web-based applications, site scraping, pages capture, SVG renderer, PDF converter and many other use cases.

PhantomJS is created by Ariya Hidayat

Solution 2

You might be able to take advantage of some of the available JavaScript unit testing tools to assemble the sort of environment you need.

For example I've found that loading env.js in rhino gives an acceptable fake browser for testing most JavaScript. It falls short on certain DOM attributes, like elements sized relative to their window, but that hasn't been a serious problem in my experience.

See also:

• http://ejohn.org/blog/bringing-the-browser-to-the-server/
• http://pivotal.github.com/jasmine/
• https://github.com/relevance/blue-ridge

Solution 3

Take a look at LoadBooster. It is a cloud-based load testing service which utilizes headless browser PhantomJS to test web sites. Disclaimer: I work for LoadBooster.

Comments

[Admin]

Thanks, I might try env.js with rhino next. This should allow me to integrate with the current java based loadtesting framwork as well. Any ideas on how it will compare to HtmlUnit? Any experience with zombie.js?

[Jonah]

From what I recall, the last time I looked at HTMLUnit it didn't have the JavaScript support I needed but that may no longer be the case. I have not tried zombie.js but it looks worth investigating, thanks for pointing that out.

[nick fox]

www.slimerjs.org though its not headless without xvfb