Find details of click and keyboard events

Question

I have noticed that there isn't much of an alternative to Problem Steps Recorder for linux, and I am going to try to make one in a shell script.

1. How can I wait until/detect when the mouse is cliked? I would like to take a screenshot at this time.
2. How can I find a description of any button or input field the mouse has clicked? Problem Steps Recorder would return something like User left-click on "close window" if I clicked the 'X' or User left-click on "close" if I clicked the "close" button.
3. How can I wait until/detect when a key is pressed and find out which key?

Solution

You could use the xinput --test-xi2 --root command (man xinput), where test-xi2 means logging input devices, and --root is to log from so called root window (a x11 specific concept). It would show you when and which key pressed in the system, mouse clicks, and even mouse motion coordinates. It solves your 1 and 3 points, you just need to parse these events.

The 2 point is trickier. You can use xev -root command which would monitor all X events in the system, including things like (dis) appearing windows, entering/exiting focus of a particular window, etc. You have to parse its output, and show a human-readable description of what is happening. Btw, the command logs pressed keys also.

A caveat: it is, obviously, a great security hole, which dates back to times when Xorg was created, and nobody needed to care. For this reason nowadays X11 is being replaced with the Wayland protocol (it is in the essence a X12, but as not backward compatible, the protocol land called another way), which WMs are implementing ATM. It isn't quick process as the protocol demands much more from WM, but Fedora 25 seem to finally have Wayland session enabled by default. KWin is also catching up. And your script not going to work on Wayland. This doesn't mean you can't do it — I think, you can. But you'd need to work closely with a particular WM of a particular DE, and the script would be not a stand alone bash script, but rather an addon to a WM.

Comments

[Nonny Moose]

Thanks for the extra information. I mainly just wanted to make this for myself and allow other people to use it, so it is fine if it works only in x11. When x11 is deprecated, then I will try to make it work with Wayland.

[Nonny Moose]

You have solved two out of my three problems. I must have phrased point 2 badly. I meant that I needed to know what text the user picked. For example, if the user clicked the Close button, I would want to say Left-click "Close", whereas xev would only return that the window closed. Do you know how I might do that?

[Hi-Angel]

@NonnyMoose well, I searched a bit, and, to be honest, I don't know. From what I know buttons in X11 system are just subwindows, and on one hand, toolkits, like GTK, Qt, or EFL, might implement the content on its own. But on the other hand, someone probably should have take care about accessibility features for blind peoples, like reading the content of buttons. But I know nothing about these APIs.

[Nonny Moose]

Thanks anyway! With this information I can still make a working version!

[WinEunuuchs2Unix]

The link is 404.

[Hi-Angel]

@WinEunuuchs2Unix thx, fixed. Hmm, what was I even thinking when using some site I've never seen as a reference for the man page.