using OR and NOT in solr query

Question

I'm working on a solr query similar to the following:

((myField:superneat AND myOtherField:somethingElse) OR NOT myField:superneat)

When running this, no results are returned. Using criteria on either side of the OR NOT returns results that I'd expect - they are just not working well together. In the case that myField matches superneat, I'm intending to also ensure that myOtherField is set to somethingElse, but if myField is not superneat, include it in the results.

Can someone explain why solr is not returning results for this kind of query? Should the query be restructured somehow - or is there a different way in which solr can be used to achieve the desired result?

Solution 1

I don't know why that doesn't work, but this one is logically equivalent and it does work:

-(myField:superneat AND -myOtherField:somethingElse)

Maybe it has something to do with defining the same field twice in the query...

Try asking in the solr-user group, then post back here the final answer!

Solution 2

Solr currently checks for a "pure negative" query and inserts *:* (which matches all documents) so that it works correctly.

-foo is transformed by solr into (*:* -foo)

The big caveat is that Solr only checks to see if the top level query is a pure negative query! So this means that a query like bar OR (-foo) is not changed since the pure negative query is in a sub-clause of the top level query. You need to transform this query yourself into bar OR (*:* -foo)

You may check the solr query explanation to verify the query transformation:

?q=-title:foo&debug=query

is transformed to

(+(-title:foo +MatchAllDocsQuery(*:*))

Solution 3

Putting together comments from a couple different answers here, in the Solr docs and on the other SO question, I found that the following syntax produces the correct result for my use case

(my_field=my_value or my_field is null):

(my_field:"my_value" OR (*:* NOT my_field:*))

This works for solr 4.1.0. This is slightly different than the use case in the OP; but, I thought that others would find it useful.

Solution 4

You can find the follow up to the solr-user group on: solr user mailling list

The prevailing thought is that the NOT operator may only be used to remove results from a query - not just exclude things out of the entire dataset. I happen to like the syntax you suggested mausch - thanks!

Solution 5

Just to add another unexpected case, here is query that wasn't returning expected results:

*:* AND ( ( field_a:foo AND field_b:bar ) OR !field_b:bar )

field_b in my case is something I perform faceting on, and needed to target the query term "foo" only on that type (bar)

I had to insert another *:* after the or condition to get this to work, like so:

*:* AND ( ( field_a:foo AND field_b:bar ) OR ( *:* AND !field_b:bar ) )

edit: this is in solr 6.6.3

View more solutions

Comments

[Admin]

Thank you for your help! This does indeed work - and I have posed this to the solr-user group. I'll post any useful things I hear from them here.

[Yorick Sijsling]

Note that -myField:superneat OR myOtherField:somethingElse would also be the same and is slightly simpler.

[Mauricio Scheffer]

@YorickSijsling the point is that even though logically equivalent, Solr sometimes doesn't cope very well with purely negative queries like the one the OP posted or the one you posted.

[terrance.a.snyder]

@Mauricio Scheffer - I would question that entirely. Could you explain more on how it doesn't cope very well? We run rather complex conditionals here and found it copes very well with billions of documents.

[Mauricio Scheffer]

@terrance.a.snyder "complex conditionals" != "purely negative queries". Also, things might have improved in recent versions of Solr, I haven't checked.

[Sebastien Lorber]

In Solr 4.1 I get this problem on date ranges, like you answered here: stackoverflow.com/questions/1343794/…

[Ryan]

I found that as of Solr 3.6, if I wanted to do a negative query OR'd with another condition I needed to do "( -myfield:myValue ) OR myField:otherValue". If I did not include spaces within the parenthesis, it did not parse. I think that is corrected in 4+.

[Ryan Shelley]

Ran into exactly this scenario today in Solr 5, and this suggestion works.

[Peter Dixon-Moses]

edismax handles nested pure negative queries correctly, right? Has there been any discussion about patching the Lucene query parser to support these in the same way?