How to simulate mouse clicks in QML?

Question

I'm trying to send a QMouseEvent to the QML objects being currently displayed. The QApplication::sendEvent() always returns false meaning that my event did not get handled; but I don't see why.
Maybe I'm sending the event to the wrong object? Where should I send it to? I also played around with QGraphicsSceneMouseEvent instead of QMouseEvent but had no luck either.

I tried stepping through the event code with the debugger but it is too complex for me to see why it's not working.

Background

I'm working on a piece of software that will be controlled via a simple touch screen. I get the touch events via ethernet and I want to synthesize mouse click events from them. This way the software will be controlled on the target device in the same way as on a developer PC.

Update

1. As noted by fejd, the click code was executed before QApplication::Exec(), so I moved it into a timer handler that will be triggered while exec() is running.
2. Added Windows-specific code that works as expected.
3. Added some more attempts in Qt none of which works no matter whether sendEvent() returns true or false.

So far I have this:

main.cpp

#include <QtGui/QApplication>
#include "qmlapplicationviewer.h"

#include "clicksimulator.h"

#include <QTimer>

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);

    QmlApplicationViewer viewer;
    viewer.setOrientation(QmlApplicationViewer::ScreenOrientationAuto);
    viewer.setMainQmlFile(QLatin1String("qml/qmlClickSimulator/main.qml"));
    viewer.showMaximized();

    ClickSimulator sim(&viewer);
    QTimer timer;
    sim.connect(&timer, SIGNAL(timeout()), SLOT(click()));
    timer.start(100);

    return app.exec();
}

clicksimulator.h

#ifndef CLICKSIMULATOR_H
#define CLICKSIMULATOR_H

#include <QObject>

class QmlApplicationViewer;

class ClickSimulator : public QObject
{
    Q_OBJECT
    QmlApplicationViewer* m_viewer;
public:
    explicit ClickSimulator(QmlApplicationViewer* viewer, QObject *parent = 0);

public slots:
    void click();    
};

#endif // CLICKSIMULATOR_H

clicksimulator.cpp

#include "clicksimulator.h"
#include "qmlapplicationviewer.h"

#include <QMouseEvent>
#include <QDebug>
#include <QGraphicsSceneMouseEvent>
#include <QApplication>
#include <QGraphicsScene>
#include <QTest>

#define _WIN32_WINNT 0x0501
#define WINVER 0x0501
#include "Windows.h"


ClickSimulator::ClickSimulator(QmlApplicationViewer* viewer, QObject *parent) :
QObject(parent)
, m_viewer(viewer)
{
}

void ClickSimulator::click()
{
    if (NULL != m_viewer)
    {
        const int x = qrand() % 500 + 100, y = qrand() % 500 + 100;

        {
           QMouseEvent pressEvent(QEvent::MouseButtonPress, QPoint(x, y), Qt::LeftButton, Qt::LeftButton, Qt::NoModifier);
//            QMouseEvent pressEvent(
//                QEvent::MouseButtonPress, 
//                QPoint(x, y),
//                Qt::LeftButton, Qt::LeftButton, Qt::NoModifier);
           const bool isSent = QApplication::sendEvent(m_viewer->scene(), &pressEvent);
           qDebug() << "'Press' at (" << x << "," << y << ") successful? " << isSent;
        }

        {
            QGraphicsSceneMouseEvent pressEvent(QEvent::GraphicsSceneMousePress);
            pressEvent.setScenePos(QPointF(x, y));
            pressEvent.setButton(Qt::LeftButton);
            pressEvent.setButtons(Qt::LeftButton);

            QGraphicsItem* item = m_viewer->itemAt(x, y);
            const bool isSent = m_viewer->scene()->sendEvent(item, &pressEvent);
            //const bool isSent = QApplication::sendEvent(m_viewer->scene(), &pressEvent);
            qDebug() << "'Press' at (" << x << "," << y << ") successful? " << isSent;
        }

        // This platform specific code works...
        {
            const double fScreenWidth = ::GetSystemMetrics( SM_CXSCREEN )-1; 
            const double fScreenHeight = ::GetSystemMetrics( SM_CYSCREEN )-1; 
            const double fx = x*(65535.0f/fScreenWidth);
            const double fy = y*(65535.0f/fScreenHeight);

            INPUT inp[3];
            inp[0].type = INPUT_MOUSE;

            MOUSEINPUT & mi = inp[0].mi;
            mi.dx = fx;
            mi.dy = fy;
            mi.mouseData = 0;
            mi.dwFlags = MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
            mi.time = 0;
            mi.dwExtraInfo = 0;

            inp[1] = inp[0];
            inp[1].mi.dwFlags = MOUSEEVENTF_LEFTDOWN;

            inp[2] = inp[0];
            inp[2].mi.dwFlags = MOUSEEVENTF_LEFTUP;

            SendInput(3, inp, sizeof(INPUT));
        }
    }
}

main.qml

import QtQuick 1.0

Rectangle {
    width: 360
    height: 360
    Text {
        id: text1
        text: qsTr("This text is placed at the click coordinates")
    }

    MouseArea {
        id: mousearea1
        anchors.fill: parent
        onClicked: {
        console.log("click at " + mouse.x + ", " + mouse.y);
            text1.pos.x = mouse.x;
            text1.pos.y = mouse.y;
        }
    }
}

output

'Press' at ( 147 , 244 ) successful?  false 
'Press' at ( 147 , 244 ) successful?  true 

Solution 1

Since you send the event before app.exec(), I don't think the main event loop has been started . You could try postEvent instead, though that might fail as well if exec() clears the event queue before it starts. In that case, perhaps you can post it somewhere after exec()?

Update: Got it working now by looking at the QDeclarativeMouseArea autotests. What was missing was the release event. This worked for me:

QGraphicsSceneMouseEvent pressEvent(QEvent::GraphicsSceneMousePress);
pressEvent.setScenePos(QPointF(x, y));
pressEvent.setButton(Qt::LeftButton);
pressEvent.setButtons(Qt::LeftButton);
QApplication::sendEvent(m_viewer->scene(), &pressEvent);

QGraphicsSceneMouseEvent releaseEvent(QEvent::GraphicsSceneMouseRelease);
releaseEvent.setScenePos(QPointF(x, y));
releaseEvent.setButton(Qt::LeftButton);
releaseEvent.setButtons(Qt::LeftButton);
QApplication::sendEvent(m_viewer->scene(), &releaseEvent);

What was a bit odd was that onPressed in the QML file did not get called after the press event - only after the release event was sent as well.

Solution 2

The accepted answer works, but there is a better way. Just use QGraphicScene's

bool sendEvent(QGraphicsItem* item, QEvent* event)

function. If you know which item to send the event to.

Comments

[foraidt]

Good point regarding exec(). Moving the code into a timer-handler did not work either, though. I updated the question with my current test code.

[fejd]

@foraidt I've updated the answer. Tested on a Mac though, hopefully the behavior is the same on Windows.