Spring - mongodb - aggregation - The 'cursor' option is required

Question

Executing the following aggregation pipeline:

public void getMostLikedItems () {
        UnwindOperation unwind = Aggregation.unwind("favoriteItems");
        GroupOperation group = Aggregation.group("favoriteItems").count().as("likes");
        SortOperation sort = Aggregation.sort(Sort.Direction.DESC, "likes");

        Aggregation aggregation = newAggregation(unwind, group, sort);
        DBObject result = mongoTemplate.aggregate(aggregation, "users", LikedItem.class).getRawResults();
}

throws the following exception:

com.mongodb.MongoCommandException: Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }

I don't understand what is meant by cursor option here. Where should this option be configured?

EDIT Here is a sample user document

{
  "_id": "5a6df13552f42a34dcca9aa6",
  "username": "user1",
  "password": "$2a$10$p0OXq5PPa41j1e4iPcGZHuWjoKJ983sieS/ovFI.cVX5Whwj21WYi",
  "favoriteItems": [
    {
      "_id": "5a0c6b2dfd3eb67969316d6d",
      "name": "item1",
      "city": "Rabat"
    },
    {
      "_id": "5a0c680afd3eb67969316d0b",
      "name": "item2",
      "city": "Rabat"
    }
  ]
}

Solution 1

From the docs.

MongoDB 3.4 deprecates the use of aggregate command without the cursor option, unless the pipeline includes the explain option. When returning aggregation results inline using the aggregate command, specify the cursor option using the default batch size cursor: {} or specify the batch size in the cursor option cursor: { batchSize: }.

You can pass batchSize with AggregationOptions in Spring Mongo 2.x version

Aggregation aggregation = newAggregation(unwind, group).withOptions(newAggregationOptions().cursorBatchSize(100).build());

With default batch size

Aggregation aggregation = newAggregation(unwind, group).withOptions(newAggregationOptions().cursor(new Document()).build());

Solution 2

'The 'cursor' option is required, except for aggregate with the explain argument'

This type of error raised in spring data when you are using incompatible versions of MongoDB and Spring-data-mongo.

Though you can get rawResults with explain, cursor arguments.

Aggregation aggregation = Aggregation.newAggregation(group).withOptions( new AggregationOptions(allowDiskUse, explain, cursor));

//try with .withOptions( new AggregationOptions(true,false,new Document()));

Passing by commented Arguments you will get result in rawResult but it will not be mapped in given outType.class.

To get mapped result you have to download right dependency of spring-data version according to your MongoDb version.

EDIT

I have used Spring version 5.0.3 and Spring-data-mongoDB version 2.0.3 It is working Fine.

Solution 3

You can provide outputmode as cursor as providing a cursor is mandatory

List<DBObject> list = new ArrayList<DBObject>();
list.add(unwind.toDBObject(Aggregation.DEFAULT_CONTEXT));
list.add(group.toDBObject(Aggregation.DEFAULT_CONTEXT));
list.add(sort.toDBObject(Aggregation.DEFAULT_CONTEXT));

DBCollection col = mongoTemplate.getCollection("users");

Cursor cursor = col.aggregate(list, AggregationOptions.builder().allowDiskUse(true).outputMode(OutputMode.CURSOR).build());

List<AggregationResultVO> result = new ArrayList<AggregationResultVO>();

while(cursor.hasNext()) {
     DBObject object = cursor.next();
     result.add(new AggregationResultVO(object.get("aggregationResultId").toString()));
}

Comments

[Blank]

My spring boot parent project is version 1.5.9. Can I upgrade spring data mongodb to version 2? sorry if this is outside the context of the question

[s7vr]

Np. Do you see AggregationOptions class ? You can <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-mongodb</artifactId> <version>2.0.2.RELEASE</version> </dependency> in your Pom dependencies to download 2.0 version. Btw I'll check what verison of spring mongo does spring boot 1.5.9 provide.

[s7vr]

1.5.9 boot references 1.10.9 spring mongo version which has AggregationOptions class. So you can try Aggregation aggregation = newAggregation(unwind, group).withOptions(newAggregationOptions().cursor(new BasicDBObject());

[Blank]

i just tried it. the options need to be built before passing to withOptions() because this function takes AggregationOptions and not AggregationOptions.Builder. Thus, i did this AggregationOptions options = newAggregationOptions().cursor(new BasicDBObject()).build(); then Aggregation aggregation = newAggregation(unwind, group).withOptions(options); then System.out.println(mongoTemplate.aggregate(aggregation, "users", LikedItem.class).getRawResults()); but it returns { "cursor" : { "firstBatch" : [ ] , "id" : 0 , "ns" : "ShopDB.users"} , "ok" : 1.0}

[s7vr]

I see. You can also try Aggregation aggregation = newAggregation(unwind, group).withOptions(newAggregationOptions().cursor(new BasicDBObject()).build()); Did you try setting batchsize ? You dont see any results do you ?

[Blank]

No result. I think maybe new BasicDBObject() has default batchsize of zero

[s7vr]

I dont think so. May be your query is not returning any results. Did you get a chance to try with setting some batch size ? Something like Aggregation aggregation = newAggregation(unwind, group).withOptions(newAggregationOptions().cursorBatchSize(1‌​0).build());

[Blank]

Not really, after i upgraded my spring mongo jar to 2.0.3, my application throws other exceptions that i cant deal with at the moment so i rollbacked to the default one spring-boot-starter-data-mongodb. what i tried instead is newAggregationOptions().cursor(new BasicDBObject("batchSize", 100)).build() but no results still.

[s7vr]

Oh sorry. Can you output aggregation.toString() to console and copy & paste the generated query into shell and see if it returns anything ? Add some sample document to the post and I can also try at my end.

[Blank]

This is what aggregate generates { "aggregate" : "__collection__" , "pipeline" : [ { "$unwind" : "$favoriteItems"} , { "$group" : { "_id" : "$favoriteItems" , "likes" : { "$sum" : 1}}}] , "cursor" : { }} i noticed that the collection name is not users. I tried it in the shell and i get 'SyntaxError: missing ; before statement @(shell):1:14' for both __collection__ and users

[Blank]

Also, I edited the post to include a sample user document.

[s7vr]

That is fine. Can you try db.users.aggregate( [ { "$unwind" : "$preferredShops"} , { "$group" : { "_id" : "$preferredShops" , "likes" : { "$sum" : 1}}}], {"cursor" : { "batchSize" : 100}} ) in shell ? Update works for me in shell with document you've provided. Just needed to change to $favoriteItems

[Blank]

Yes the aggregate function works for me too when triggered from the shell. The issue is with its java translation

[s7vr]

Indeed the cursor option is not working in 1.10.x spring version. You can fix that by accessing the low level DBCollection and passing the aggregation stages with AggregationOptions( mongdb pacakge)

[s7vr]

Something like DBCollection col = mongoTemplate.getCollection("users"); Cursor cursor = col.aggregate(Arrays.asList(unwind.toDBObject(Aggregation.DE‌​FAULT_CONTEXT), group.toDBObject(Aggregation.DEFAULT_CONTEXT), sort.toDBObject(Aggregation.DEFAULT_CONTEXT)), AggregationOptions.builder().batchSize(100).build()); while(cursor.hasNext()) { DBObject dbObject = (DBObject) cursor.next(); System.out.println(dbObject); }

[gregfqt]

In boot 1.5 / data-mongo 1.10, mongoTemplate.aggregate(...).getRawResults() returns a DBObject which maps this structure : { "cursor" : { "firstBatch" : [ {"_id" : ObjectId("...")}, {"_id" : ObjectId("...")}, ], "id" : NumberLong(0), "ns" : "..." }, "ok" : 1.0, "operationTime" : ... }. You have to walk through cursor.firstBatch using DBObject.get(...) to get the actual query results, Spring cannot map reults from the cursor. (That's hardly readable, sorry)