Retrieve multiple messages from SQS

Question

I have multiple messages in SQS. The following code always returns only one, even if there are dozens visible (not in flight). setMaxNumberOfMessages I thought would allow multiple to be consumed at once .. have i misunderstood this?

 CreateQueueRequest createQueueRequest = new CreateQueueRequest().withQueueName(queueName);
 String queueUrl = sqs.createQueue(createQueueRequest).getQueueUrl();
 ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(queueUrl);
 receiveMessageRequest.setMaxNumberOfMessages(10);
 List<Message> messages = sqs.receiveMessage(receiveMessageRequest).getMessages();
 for (Message message : messages) {
      // i'm a message from SQS
 }

I've also tried using withMaxNumberOfMessages without any such luck:

 receiveMessageRequest.withMaxNumberOfMessages(10);

How do I know there are messages in the queue? More than 1?

 Set<String> attrs = new HashSet<String>();
 attrs.add("ApproximateNumberOfMessages");
 CreateQueueRequest createQueueRequest = new CreateQueueRequest().withQueueName(queueName);
 GetQueueAttributesRequest a = new GetQueueAttributesRequest().withQueueUrl(sqs.createQueue(createQueueRequest).getQueueUrl()).withAttributeNames(attrs);
 Map<String,String> result = sqs.getQueueAttributes(a).getAttributes();
 int num = Integer.parseInt(result.get("ApproximateNumberOfMessages"));

The above always is run prior and gives me an int that is >1

Thanks for your input

Solution 1

AWS API Reference Guide: Query/QueryReceiveMessage

Due to the distributed nature of the queue, a weighted random set of machines is sampled on a ReceiveMessage call. That means only the messages on the sampled machines are returned. If the number of messages in the queue is small (less than 1000), it is likely you will get fewer messages than you requested per ReceiveMessage call. If the number of messages in the queue is extremely small, you might not receive any messages in a particular ReceiveMessage response; in which case you should repeat the request.

and

MaxNumberOfMessages: Maximum number of messages to return. SQS never returns more messages than this value but might return fewer.

Solution 2

There is a comprehensive explanation for this (arguably rather idiosyncratic) behaviour in the SQS reference documentation.

SQS stores copies of messages on multiple servers and receive message requests are made to these servers with one of two possible strategies,

• Short Polling : The default behaviour, only a subset of the servers (based on a weighted random distribution) are queried.
• Long Polling : Enabled by setting the WaitTimeSeconds attribute to a non-zero value, all of the servers are queried.

In practice, for my limited tests, I always seem to get one message with short polling just as you did.

Solution 3

I had the same problem. What is your Receive Message Wait Time for your queue set to? When mine was at 0, it only returned 1 message even if there were 8 in the queue. When I increased the Receive Message Wait Time, then I got all of them. Seems kind of buggy to me.

Solution 4

I was just trying the same and with the help of these two attributes setMaxNumberOfMessages and setWaitTimeSeconds i was able to get 10 messages.

ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(myQueueUrl);
                      receiveMessageRequest.setMaxNumberOfMessages(10);
                      receiveMessageRequest.setWaitTimeSeconds(20);

Snapshot of o/p:

Receiving messages from TestQueue.
Number of messages:10
Message
MessageId:     31a7c669-1f0c-4bf1-b18b-c7fa31f4e82d 
...

Solution 5

receiveMessageRequest.withMaxNumberOfMessages(10);

Just to be clear, the more practical use of this would be to add to your constructor like this:

ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(queueUrl).withMaxNumberOfMessages(10);

Otherwise, you might as well just do:

receiveMessageRequest.setMaxNumberOfMessages(10);

That being said, changing this won't help the original problem.

View more solutions