Record Actions using Selenium

Question

I have a semi-vague question to ask about Selenium. I've discovered a few different ways to perform actions using the FirefoxDriver. What I need to do is repeat actions that a user performs on a web page (clicking a link, checking a checkbox, etc.). Is there any method or combination of methods that allows me to "record" the user's actions? Here is what I have so far to perform actions (you'll notice I've tried using the WebDriverBackedSelenium and Actions classes to perform actions)

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriverBackedSelenium;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.interactions.Actions;
import org.openqa.selenium.interactions.Action;

public class MyReplayer {
    public static void main(String[] args) throws Exception {
        // The Firefox driver supports javascript 
        FirefoxDriver driver = new FirefoxDriver();

        driver.get("http://www.cs.umd.edu");

        List<WebElement> elements = driver.findElements(By.tagName("a"));
        //WebDriverBackedSelenium driverBacked = new WebDriverBackedSelenium(driver,        "http://www.cs.umd.edu");
        Actions builder = new Actions(driver);    
        Action clickLink = builder.click(elements.get(100)).build();
        clickLink.perform();
        //driverBacked.click("document.getElementsByTagName('a')[100]");
     }
}

Solution 1

I came across Huxley. It allows recording and playback of user actions. I found this question in search of how they did it, but had to resort to source code.

Lines 98-154 of huxley/run.py define the record function. It uses webdirvier to execute some js on the page which adds some event listeners. It also adds a function to return the events.

(function() {
var events = [];
window.addEventListener('click', function (e) { events.push([Date.now(), 'click',  [e.clientX, e.clientY]]); }, true);
window.addEventListener('keyup', function (e) { events.push([Date.now(), 'keyup', String.fromCharCode(e.keyCode)]); }, true);
window._getHuxleyEvents = function() { return events; };
})();

To read the events the js function is called

events = d.execute_script('return window._getHuxleyEvents();')

Then the events are stored in a way that seems application specific.

Sorry, I do not have Java code. I hope this helps.

Solution 2

You can use the Selenium IDE Addon for Firefox and export the generated test for Webdriver. It doesn't specifically say FirefoxDriver, but the methods of the interface look similar to what you posted. I hope this helps.

Solution 3

I am currently working on a project that does something like this: https://github.com/hristo-vrigazov/selenium-record-replay It works by putting a proxy between the browser and the application, and injecting javascript which listens for actions that you have defined. See for example https://github.com/hristo-vrigazov/selenium-record-replay/blob/master/terminator-cli/src/main/java/browser/Main.java#L74

        RecordBrowserBase recordBrowserBase = new ChromeRecordBrowser(pathToChromedriver, pathToJSInjectionFile);
        try {
            recordBrowserBase.record(baseUrl);
            System.out.println("Press Enter when finished recording");
            System.in.read();
            recordBrowserBase.dumpActions(outputFile);
        } catch (IOException | InterruptedException | URISyntaxException e) {
            e.printStackTrace();
        }

        recordBrowserBase.cleanUp();

        System.exit(0);

The project is still in a very early stage, but it can be used even now. Currently only Chrome is supported, but I will soon add other browsers as well.

Disclaimer: I am the creator and maintainer of the project

Comments

[Admin]

Do you know how the Selenium IDE is able to record the user's actions? That's the functionality I am looking to implement myself but I don't know of any methods to help me do that.

[ma cılay]

I dont know how Selenium IDE does it. But i'd think that they catch change events in the DOM of the page you're looking at. I dont know how much they are integrated to the Browser GUI but that could be happening too. Most of that can be found out on the server side too, but finding out if a checkbox was checked without sending the page to the server can only happen on client (or you'll have to make ajax calls for everything)

[Łukasƨ Fronczyk]

Won't work for an element if any it its existing event handlers call stopImmediatePropagation().