Record and replay Javascript

javascript dom-events

24,329

Solution 1

Replaying user actions with just Javascript is a complex problem.

First of all, you can't move mouse cursor, you can't emulate mouseover/hovers also. So there goes away a big part of user interactions with a page.

Second of all, actions, once recorded, for most of the time they have to be replayed in different environment than they were recorded in the first place. I mean you can replay the actions on screen with smaller resolutions, different client browser, different content served based on replaying browser cookies etc.

If you take a time to study available services that enable you to record website visitors actions ( http://clicktale.com, http://userfly.com/ to name a few), you'll see that none of them are capable of fully replaying users actions, especially when it comes to mouseovers, ajax, complex JS widgets.

As to your question for detecting changes made to the DOM - as Chris Biscardi stated in his answer, there are mutation events, that are designed for that. However, keep in mind, that they are not implemented in every browser. Namely, the IE doesn't support them (they will be supported as of IE 9, according to this blog entry on msdn http://blogs.msdn.com/b/ie/archive/2010/03/26/dom-level-3-events-support-in-ie9.aspx).

Relying on those events may be suitable for you, depending on use case.

As to "better more simple way to store all events". There are other ways (syntax wise), of listening for events of your choice, however handling (= storing) them can't be handled in simple way, unless you want to serialize whole event objects which wouldn't be a good idea, if you were to send information about those event to some server to store them. You have to be aware of the fact, that there are massive amount of events popping around while using website, hence massive amount of potential data to be stored/send to the server.

I hope I made myself clear and you find some of those information useful. I myself have been involved in project that aimed to do what you're trying to achive, so I know how complicated can it get once you start digging into the subject.

Solution 2

I became curious by this question and implemented a proof of concept here

https://codesandbox.io/s/jquery-playground-y46pv?fontsize=14&hidenavigation=1&theme=dark

Using the demo

Try pressing record, clicking around, press record again and then click play.
Play creates an <iframe>, injects the original HTML and replays the user events.
To change zoom change the REPLAY_SCALE variable in the source code.
To change the playback speed change the SPEED variable in the source code.
NB I only tested the jsbin on chrome 56.

Implementation details:

It supports mousemove, click and typing. I omitted others (scroll, window resizing, hover, focus etc etc) but should be easily extensible.
The event listeners bypass any event.stopPropagation() by using capture when listening for events on the document.
Displaying playback in a different resolution is done using zoom CSS3.
A transparent canvas could be overlaid to draw the trace lines of the mouse. I use just a simple div so no trace lines.

Considerations:

Imagining we are capturing user events on a real website. Since the page served could change between now and the playback we can't rely on the client's server when replaying the recording in the iframe. Instead we have to snapshot the html, all ajax requests and resource requests made during the recording. In the demo I only snapshot the HTML for simplicity. However in practice, all extra requests would have to be stored on the server in realtime as they are downloaded on the client page. Furthermore, during playback it is important that the requests are played back with the same timeline that they were perceived by the user. To simulate the request timeline, the offset and duration of each request must also be stored. Uploading all page requests as they are downloaded on the client will slow down the client page. One way to optimize this uploading could be to md5 hash the contents of the request before they are uploaded, if the md5 hash is already present on the server, the request data need not be reuploaded. Furthermore, the session of one user can leverage the request data uploaded by another user using this hashing method.
Careful consideration will be needed when uploading all the events. Since lots of events will be generated, this means lots of data. Perhaps some compression of the events could be made e.g. losing some of the less important mousemove events. An upload request should not be made per event to minimize number of requests. The events should be buffered until a buffer size or timeout is reached before each batch of events is uploaded. A timeout should be used as the user could close the page at any point thus losing some events.
During playback outgoing POST requests should be mocked to prevent duplicating events elsewhere.
During playback the user agent should be spoofed but this may be unreliable in rendering the original display.
The custom recording code could conflict with client code. e.g. jquery. Namespacing will be required to avoid this.
There might be some edge cases where typing and clicking may not reproduce the same resulting HTML as seen in the original e.g. random numbers, date times. Mutation observers may be required to observe HTML changes, although not supported in all browsers. Screenshots could come in useful here but might prove OTT.

Solution 3

Record

Save the initial DOM of the page, remove the scripts from it and also you need to change all relative URLs to absolute ones.

Then, record DOM mutations and Keyboard/Mouse event.

Replay

Start with initial saved DOM, apply mutations and events using timestamp order.

In fact, clicks will not do anything because we have removed any scripts. but because we have saved the DOM changes we can replay the effect after the click.

Solution 4

I found these two solutions on github which allows your to capture the events and then replay that on a remote server.

https://github.com/ElliotNB/js-replay

and a more comprehensive solution

https://github.com/rrweb-io/rrweb

https://www.rrweb.io/#demos

Both has demos which you can try.

Solution 5

I believe you are looking for Mutation Events.

http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-mutationevents

Here are some resources for you:

http://tobiasz123.wordpress.com/2009/01/19/utilizing-mutation-events-for-automatic-and-persistent-event-attaching/

http://forum.jquery.com/topic/mutation-events-12-1-2010

https://github.com/jollytoad/jquery.mutation-events

Update:

In Response to comment, a very, very basic implementation:

//callback function
function onNodeInserted(){alert('inserted')}
//add listener to dom(in this case the body tag)
document.body.addEventListener ('DOMNodeInserted', onNodeInserted, false); 
//Add element to dom 
$('<div>test</div>').appendTo('body')

Like WTK said, you are getting yourself into complex territory.

View more solutions

24,329

Author by

Hakan

Updated on November 22, 2020

Comments

Hakan over 3 years

I know it is possible to record mouse movements, scrolling and keystrokes. But what about changes to the document? How can I record changes to the document?

Here is my try out. There must be a better more simple way to store all events?

I am thankful for all tips I can get!

<!DOCTYPE html>
<html>
<head>
<title>Record And replay javascript</title>
</head>
<body id="if_no_other_id_exist">

<div style="height:100px;background:#0F0" id="test1">click me</div>
<div style="height:100px;background:#9F9" class="test2">click me</div>
<div style="height:100px;background:#3F9" id="test3">click me</div>
<div style="height:100px;background:#F96" id="test4">click me</div>



<script src="http://code.jquery.com/jquery-latest.min.js"></script>
<script>

$(document).ready(function() {
var the_time_document_is_redy = new Date().getTime();
var the_replay = '';


$('div').live("click", function (){
var the_length_of_visit = new Date().getTime() - the_time_document_is_redy;


// check if the element that is clicked has an id
if (this.id)
{

the_replay =
the_replay
+
"setTimeout(\"$('#"
+
this.id
+
"').trigger('click')\","
+
the_length_of_visit
+
");"
;


alert (
"The following javascript will be included in the file in the replay version:\n\n"
+ 
the_replay
) // end alert

} // end if



// if it does not have an id, check if the element that is clicked has an class
else if (this.className)
{

// find the closest id to better target the element (needed in my application)
var closest_div_with_id = $(this).closest('[id]').attr('id');

the_replay =
the_replay
+
"setTimeout(\"$('#"
+
closest_div_with_id
+
" ."
+
this.className
+
"').trigger('click')\","
+
the_length_of_visit
+
");"
;


alert (
"The following javascript will be included in the file in the replay version:\n\n"
+ 
the_replay
) // end alert

} // end if

});






// fall back if there are no other id's
$('body').attr('id','if_no_other_id_exist');


// example of how it will work in the replay version
setTimeout("$('#test1').trigger('click')",10000);

});
</script>
</body>
</html>

Hakan over 12 years

Thanks for your answer. Tough I don't understand what it is, or how it works. Would you like to sho a simple "alert demo" to show whats possible with it?
Chris Biscardi over 12 years

I added some code to my answer. The Mutation events were added in DOM Level 2 and were created for the purpose of detecting changes to the DOM.
Hakan over 12 years

Thanks for your guidance. Some toughts: 1) You can record the mouse cusor related to a point in "center top" of the page. Then window size won't matter that much. But the other problems you pointed out is still there... I think I am better of continuing in the same simple way as in my example.
Chris Biscardi over 12 years

Glad to hear the implementation helped
Austin over 3 years

Would you like to enhance your answer by providing sample code
Mickey D about 2 years

Did something change? The sample playback opens an iframe and quickly flashes some movement and then the iframe disappears :-/
david_adler about 2 years

Seems to be working for me. It does destroy the playback after playing the recording.
Mickey D about 2 years

Thanks @david_adler I normally try other browsers and your comment sent me from Firefox to Chrome which it works in. Any experts know what might be causing the issue in Firefox...