Message Queue in PHP

Question

I have developed a rest api which accepts some data and store it in a message queue (Redis List). Now from redis this data is pushed into MySQL database. The problem is client needs to wait till the data is written into mysql.

I want that the client should wait till the data is written into message queue (Redis List) and the function to push the data into MySQL should execute Asynchronously. How can I do it ? My entire code base is in PHP, So I would prefer it in PHP.

I have read this but havn't tried.

Distributed queue example in PHP using Redis

I have confusion that how slave.php ( mentioned in the link) will be executed. I mean When a new message arrives in the queue, how slave.php will find this.

I dont want to use cronjob for this. Instead when a new message arrives slave.php should get executed asynchronously. How to do it?

Solution

Yes, this is very much possible using Memqueue, Redis etc.

Using Redis, this is how one can do it:

This PHP file pushes the message into queue when it gets it:

/*
    Code logic
*/
$redis->lPush("message_queue", "message 1");

A slave.php which does a "blocking pop" using brPop:

$redis = new Redis();
$redis->pconnect();

while (true) {
    list($queue, $message) = $redis->brPop(["message_queue"], 0);
    /*
    Logic to insert $message to MySQL
    */
}

Now whenever a new message arrives, the slave.php will catch it and push it to MySQL.

Don't be confused by the while(true) - above code is not running in an infinite loop. The brPop asynchronously waits till there is a new message in the queue.

I'm using PHP 5.4 and connecting to Redis 2.6 and above works fine. Better, you can have multiple slave.php files and the load gets distributed.

For more details: http://redis.io/commands/blpop

Comments

[Neel Kamal]

that statement is reagrding my doubt after reading this link redis4you.com/code.php?id=012. The system workflow is mentioned in first and second paragraph

[Neel Kamal]

Hii Manu..Thanks for your answer. It is working for me. But there is one problem. When the list is empty, slave.php waits only for few minutes after that it throws some exception. And once blocking the connection it doesn't listen even when new data arrive in the list. I have added a new question where my codes and error messages are mentioned. here is the link : stackoverflow.com/questions/20972743/… Can you please help? Thanks

[Airy]

@NeelKamal or anyone else looking for timeout other question, I have provided answer over there stackoverflow.com/questions/20972743/…