Equivalent to xdotool for Wayland

Question

I'm looking for a solution working in Wayland to get mouse position, move it and click inside a bash script, all things xdotool can do for X server.

xdotool getmouselocation

is still working, but

xdotool mousemove 
xdotool click 

have no impact on the mouse cursor in GNOME Wayland session.

Solution 1

This is because such features have been explicitly removed from Wayland for security reasons. The major concerns were reading other programs input and allowing fake input to be sent to other programs which would allow different attack vectors.

Some window-managers might implement some sort of macro feature in the future but as of now there is no such feature that I know of.

Implementing this would mean to implement it for each of the different window managers which surely will take still a while.

I suggest using Xorg instead of Wayland for now if you need to use such features.

Solution 2

There is the ydotool package for wayland:

Package ydotool

Generic command-line automation tool (no X!)

https://github.com/ReimuNotMoe/ydotool

Performs some of the functions of xdotool(1) without requiring X11 - however, it generally requires root permission (to open /dev/uinput)

Currently implemented command(s):

• type - Type a string
• key - Press keys
• mousemove - Move mouse pointer to absolute position
• mousemove_relative - Move mouse pointer to relative position
• mouseup - Generate mouse up event
• mousedown - Generate mouse down event
• click - Click on mouse buttons
• recorder - Record/replay input events

N.B. optionally, you can start the ydotoold daemon with:

• systemctl enable ydotool
• systemctl start ydotool

Version: 0.1.9

Solution 3

evemu from the evemu-tools package can emulate devices like mouse, touchpads and keyboards in wayland. It cannot "read" what is happening on the screen but can easily move & click and "blindly" interact with the session.

  sudo evemu-describe # list devices

if your mouse is /dev/input/event5 from the output of above command the following will move mouse 50 pixels xy then right-press & let go of button:

    sudo evemu-event /dev/input/event5 --type EV_REL --code REL_X --value 50
    sudo evemu-event /dev/input/event5 --type EV_REL --code REL_Y --value 50 
    sudo evemu-event /dev/input/event5 --type EV_KEY --code BTN_RIGHT --value 1 
    sudo evemu-event /dev/input/event5 --type EV_KEY --code BTN_RIGHT --value 0

You can also record a particular sequence and re-play it:

# press ctrl-c to stop recording
sudo evemu-record /dev/input/event5 /tmp/mouse-sequence
# to replay session
sudo evemu-play /tmp/mouse-sequence

Solution 4

There is the wtype package for Wayland. It's available on apt repos on Ubuntu 22.04.

Seems a simpler approach (no need for ydotoold/systemd service), but for the time being, mutter (GNOME compositor) doesn't work with it.

Check also Evemu, a freedesktop project that records and replays device descriptions and events.

View more solutions

Comments

[Ben Creasy]

See also superuser.com/questions/1032270/… and unix.stackexchange.com/questions/381831/…

[Ben Creasy]

I don't know if you are aware of any of the discussions, but there was a RFC on a mailing list: RFC: Interface for injection of input events and Add an API for taking screenshots and recording screencasts. It sounds like the challenges could be addressed with some additional hardening work.

[Fabian Röling]

Is this still up to date? If no click/type automation is possible, it would be an absolute blocker against Wayland for me. (Also, it's never great when someone tells the user what they're allowed to do on their own computer. This is not Windows.)

[Arne Babenhauserheide]

So, re ydotool, for security reasons, automation on wayland now requires a daemon with root privileges that affects all users, instead of the fake input tool being contained to a specific x11-session of a specific user. The issue for taking screenshots is still open 5 years later: gitlab.freedesktop.org/wayland/wayland/-/issues/32