Java Robot class simulating human mouse movement

11,988

Solution 1

There are a few things to consider if you want to make the artificial movement natural, I think:

  1. Human mouse movement is usually in a slight arc because the mouse hand pivots around the wrist. Also that arc is more pronounced for horizontal movements than vertical.
  2. Humans tend to go in the general direction, often overshoot the target and then go back to the actual target.
  3. Initial speed towards the target is quite fast (hence the aforementioned overshoot) and then a bit slower for precise targeting. However, if the cursor is close to the target initially the quick move towards it doesn't happen (and neither does the overshoot).

This is a bit complex to formulate in algorithms, though.

Solution 2

For anyone in the future: I developed a library for Java, that mimics human mouse movement. The noise/jaggedness in movement, sinusoidal arcs, overshooting the position a bit, etc. Plus the library is written with extension and configuration possibilities in mind, so anyone can fine tune it, if the default solution is not matching the case. Available from Maven Central now.

https://github.com/JoonasVali/NaturalMouseMotion

Solution 3

Take a look in this example that I wrote. You can improve this to simulate what Joey said. I wrote it very fast and there are lots of things that can be improved (algorithm and class design). Note that I only deal with left to right movements.

import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Point;
import java.awt.Robot;

public class MouseMoving {

    public static void main(String[] args) {
        new MouseMoving().execute();
    }

    public void execute() {
        new Thread( new MouseMoveThread( 100, 50, 50, 10 ) ).start();
    }

    private class MouseMoveThread implements Runnable {

        private Robot robot;
        private int startX;
        private int startY;
        private int currentX;
        private int currentY;
        private int xAmount;
        private int yAmount;
        private int xAmountPerIteration;
        private int yAmountPerIteration;
        private int numberOfIterations;
        private long timeToSleep;

        public MouseMoveThread( int xAmount, int yAmount,
                int numberOfIterations, long timeToSleep ) {

            this.xAmount = xAmount;
            this.yAmount = yAmount;
            this.numberOfIterations = numberOfIterations;
            this.timeToSleep = timeToSleep;

            try {

                robot = new Robot();

                Point startLocation = MouseInfo.getPointerInfo().getLocation();
                startX = startLocation.x;
                startY = startLocation.y;

            } catch ( AWTException exc ) {
                exc.printStackTrace();
            }

        }

        @Override
        public void run() {

            currentX = startX;
            currentY = startY;

            xAmountPerIteration = xAmount / numberOfIterations;
            yAmountPerIteration = yAmount / numberOfIterations;

            while ( currentX < startX + xAmount &&
                    currentY < startY + yAmount ) {

                currentX += xAmountPerIteration;
                currentY += yAmountPerIteration;

                robot.mouseMove( currentX, currentY );

                try {
                    Thread.sleep( timeToSleep );
                } catch ( InterruptedException exc ) {
                    exc.printStackTrace();
                }

            }

        }

    }

}
Share:
11,988
Shawn Lien
Author by

Shawn Lien

Updated on June 28, 2022

Comments

  • Shawn Lien
    Shawn Lien over 1 year

    I am working on a project about remote control, send conrdinate x and y of cursor from client to server.

    But

    robot.mouseMove(x,y);
    

    will only move the cursor to the particular point without moving the cursor form origional point

    I have find this simple algorthim to simulate the continuing movement of mouse

    for (int i=0; i<100; i++){
       int x = ((end_x * i)/100) + (start_x*(100-i)/100);
     int y = ((end_y * i)/100) + (start_y*(100-i)/100);
     robot.mouseMove(x,y);
    } 
    

    But this algorthim still too simple, it just move from one point to other point slowly, which still unlike human behave.

    I have read some open soruce code about remote control from web, and I find this project http://code.google.com/p/java-remote-control/ is using the method call MosueMovement from MouseListener class, which they use to perform the "dragging".

    I like to know is any one know the better way of doing this?

  • Shawn Lien
    Shawn Lien about 11 years
    Thanks for adive, I am working on the project to remote control from Android phone to desktop, most people done it well but looks use java to write the function is little bit difficult, I will keep work on this
  • Shawn Lien
    Shawn Lien about 11 years
    Thanks for the code, I will keep work and see what I can do on it
  • Joonas Vali
    Joonas Vali about 5 years
    Hmm, why the deletion request and downvote, if you don't mind me asking? (To whoever requested this.)